[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:06 dist-test-slave-p6hz krb5kdc[13806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:06 dist-test-slave-p6hz krb5kdc[13806](info): set up 2 sockets
Sep 27 07:00:06 dist-test-slave-p6hz krb5kdc[13806](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260927 07:00:07.719632 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:07 dist-test-slave-p6hz krb5kdc[13806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:07.881214 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:39955
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:45789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:39955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:08.198957 13819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:08.199338 13819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:08.199474 13819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:08.206696 13819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:08.206827 13819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:08.206913 13819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:08.206985 13819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:08.207041 13819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:08.217092 13819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:45789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:39955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:39955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:08.218489 13819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:08.219914 13819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:08.232215 13824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:08.232185 13825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:08.232239 13827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:08.232481 13819 server_base.cc:1034] running on GCE node
I20260927 07:00:08.233392 13819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:08.234920 13819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:08.236120 13819 hybrid_clock.cc:648] HybridClock initialized: now 1790492408236049 us; error 73 us; skew 500 ppm
I20260927 07:00:08.236471 13819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:08 dist-test-slave-p6hz krb5kdc[13806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492408, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:08.239621 13819 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:08.241302 13819 webserver.cc:460] Webserver started at http://127.13.121.254:36455/ using document root <none> and password file <none>
I20260927 07:00:08.242069 13819 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:08.242194 13819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:08.242501 13819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:08.245683 13819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "fd496d5634404263bb270293b7337090"
format_stamp: "Formatted at 2026-09-27 07:00:08 on dist-test-slave-p6hz"
server_key: "2355745f3f1a88a2cc62a224ebd8752b"
server_key_iv: "621cc260d5dabdbe8009a6340a797d5e"
server_key_version: "encryptionkey@0"
I20260927 07:00:08.246547 13819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "fd496d5634404263bb270293b7337090"
format_stamp: "Formatted at 2026-09-27 07:00:08 on dist-test-slave-p6hz"
server_key: "2355745f3f1a88a2cc62a224ebd8752b"
server_key_iv: "621cc260d5dabdbe8009a6340a797d5e"
server_key_version: "encryptionkey@0"
I20260927 07:00:08.252192 13819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260927 07:00:08.255990 13834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:08.257622 13819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:08.257838 13819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "fd496d5634404263bb270293b7337090"
format_stamp: "Formatted at 2026-09-27 07:00:08 on dist-test-slave-p6hz"
server_key: "2355745f3f1a88a2cc62a224ebd8752b"
server_key_iv: "621cc260d5dabdbe8009a6340a797d5e"
server_key_version: "encryptionkey@0"
I20260927 07:00:08.258049 13819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:08.277232 13819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:08.281329 13819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:08.281585 13819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:08.298432 13819 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:39955
I20260927 07:00:08.298444 13885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:39955 every 8 connection(s)
I20260927 07:00:08.300025 13819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:08.304082 13886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:08.306701 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 13819
I20260927 07:00:08.306984 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:08.308907 13799 external_mini_cluster.cc:1428] Setting key 097f5e751530a288e648880ec1f25f01
I20260927 07:00:08.316033 13886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090: Bootstrap starting.
I20260927 07:00:08.319177 13886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:08.320200 13886 log.cc:824] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:08.323240 13886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090: No bootstrap required, opened a new log
I20260927 07:00:08.330595 13886 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd496d5634404263bb270293b7337090" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39955 } }
I20260927 07:00:08.331002 13886 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:08.331104 13886 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd496d5634404263bb270293b7337090, State: Initialized, Role: FOLLOWER
I20260927 07:00:08.331815 13886 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd496d5634404263bb270293b7337090" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39955 } }
I20260927 07:00:08.332041 13886 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:08.332190 13886 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:08.332350 13886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:08.334601 13886 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd496d5634404263bb270293b7337090" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39955 } }
I20260927 07:00:08.335114 13886 leader_election.cc:302] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd496d5634404263bb270293b7337090; no voters:
I20260927 07:00:08.335740 13886 leader_election.cc:288] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:08.335917 13891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [term 1 FOLLOWER]: Leader election won for term 1
Sep 27 07:00:08 dist-test-slave-p6hz krb5kdc[13806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:08.337765 13886 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:08.338841 13891 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [term 1 LEADER]: Becoming Leader. State: Replica: fd496d5634404263bb270293b7337090, State: Running, Role: LEADER
I20260927 07:00:08.339663 13891 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd496d5634404263bb270293b7337090" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39955 } }
I20260927 07:00:08.342334 13893 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd496d5634404263bb270293b7337090. Latest consensus state: current_term: 1 leader_uuid: "fd496d5634404263bb270293b7337090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd496d5634404263bb270293b7337090" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39955 } } }
I20260927 07:00:08.342732 13893 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:08.343295 13892 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd496d5634404263bb270293b7337090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd496d5634404263bb270293b7337090" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39955 } } }
I20260927 07:00:08.343573 13892 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:08.352744 13888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:08.313024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.13.121.254:39955)
0927 07:00:08.313905 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:08.313924 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:08.315107 (+ 1183us) server_negotiation.cc:407] Connection header received
0927 07:00:08.324052 (+ 8945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:08.324077 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:08.324500 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:08.324912 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:08.327944 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:08.329315 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:08.331013 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:08.331830 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:08.332011 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:08.338760 (+ 6749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:08.338774 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:08.338800 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:08.338865 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:08.343742 (+ 4877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:08.345079 (+ 1337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:08.345091 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:08.345101 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:08.345377 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:08.346026 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:08.346032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:08.346036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:08.346841 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:08.350294 (+ 3453us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:08.350597 (+ 303us) server_negotiation.cc:299] Negotiation successful
0927 07:00:08.351102 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":405,"threads_started":1}
I20260927 07:00:08.359251 13899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:08.364822 13899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:08.374426 13899 catalog_manager.cc:1347] Generated new cluster ID: a0a4625a8afe43a6992aa8936c9ba735
I20260927 07:00:08.374555 13899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:08.383435 13899 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:08.385373 13899 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:08.392680 13899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd496d5634404263bb270293b7337090: Generated new TSK 0
I20260927 07:00:08.393656 13899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:00:08.440590 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39955
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:45789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:08.764061 13913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:08.764365 13913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:08.764470 13913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:08.771885 13913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:08.772006 13913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:08.772130 13913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:08.782176 13913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:45789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39955
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:08.783517 13913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:08.784786 13913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:08.797518 13918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:08.797552 13921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:08.797582 13919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:08.798130 13913 server_base.cc:1034] running on GCE node
I20260927 07:00:08.798771 13913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:08.799765 13913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:08.800981 13913 hybrid_clock.cc:648] HybridClock initialized: now 1790492408800929 us; error 44 us; skew 500 ppm
I20260927 07:00:08.801306 13913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:08 dist-test-slave-p6hz krb5kdc[13806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492408, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:08.804426 13913 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:00:08.805830 13913 webserver.cc:460] Webserver started at http://127.13.121.193:36877/ using document root <none> and password file <none>
I20260927 07:00:08.806530 13913 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:08.806649 13913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:08.806947 13913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:08.809906 13913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "d156aa15fefb4efba56b63167830416b"
format_stamp: "Formatted at 2026-09-27 07:00:08 on dist-test-slave-p6hz"
server_key: "f2db74d25fdc9f5188579d3ffa6b7526"
server_key_iv: "b46622463fc2e376563537060cd9382d"
server_key_version: "encryptionkey@0"
I20260927 07:00:08.810667 13913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "d156aa15fefb4efba56b63167830416b"
format_stamp: "Formatted at 2026-09-27 07:00:08 on dist-test-slave-p6hz"
server_key: "f2db74d25fdc9f5188579d3ffa6b7526"
server_key_iv: "b46622463fc2e376563537060cd9382d"
server_key_version: "encryptionkey@0"
I20260927 07:00:08.815999 13913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260927 07:00:08.819535 13928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:08.820888 13913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:00:08.821045 13913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "d156aa15fefb4efba56b63167830416b"
format_stamp: "Formatted at 2026-09-27 07:00:08 on dist-test-slave-p6hz"
server_key: "f2db74d25fdc9f5188579d3ffa6b7526"
server_key_iv: "b46622463fc2e376563537060cd9382d"
server_key_version: "encryptionkey@0"
I20260927 07:00:08.821229 13913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:08.840718 13913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:08.845142 13913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:08.845419 13913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:08.846419 13913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:08.847780 13913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:08.847855 13913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:08.847965 13913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:08.848013 13913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:08.878029 13913 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:44977
I20260927 07:00:08.878067 14040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:44977 every 8 connection(s)
I20260927 07:00:08.879297 13913 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:08.887043 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 13913
I20260927 07:00:08.887326 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:08.888041 13799 external_mini_cluster.cc:1428] Setting key d8f15ef875f6b57ba27db715d0415f0c
Sep 27 07:00:08 dist-test-slave-p6hz krb5kdc[13806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492408, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:08.901504 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:08.881698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:56843 (local address 127.13.121.254:39955)
0927 07:00:08.882129 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:08.882136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:08.883107 (+ 971us) server_negotiation.cc:407] Connection header received
0927 07:00:08.884934 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:08.884938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:08.885005 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:08.885098 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:08.886795 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:08.887901 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:08.889654 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:08.890142 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:08.890249 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:08.894547 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:08.894554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:08.894560 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:08.894605 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:08.897819 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:08.898811 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:08.898817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:08.898821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:08.898986 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:08.899545 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:08.899550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:08.899554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:08.899927 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:08.900083 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:08.900862 (+ 779us) server_negotiation.cc:299] Negotiation successful
0927 07:00:08.901111 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":198,"threads_started":1}
I20260927 07:00:08.902277 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:08.882270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39955 (local address 127.13.121.193:56843)
0927 07:00:08.882944 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:08.882981 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:08.884663 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:08.885406 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:08.885415 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:08.885969 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:08.886533 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:08.886555 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:08.888079 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:08.888083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:08.889512 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:08.889525 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:08.890383 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:08.890391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:08.890690 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:08.891463 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:08.891501 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:08.894263 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:08.898006 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:08.898016 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:08.898036 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:08.898654 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:08.899169 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:08.899175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:08.899179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:08.899428 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:08.900066 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:08.900072 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:08.900467 (+ 395us) client_negotiation.cc:769] Sending connection context
0927 07:00:08.900708 (+ 241us) client_negotiation.cc:240] Negotiation successful
0927 07:00:08.901122 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":246,"threads_started":1}
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:00:08.903939 14041 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39955
I20260927 07:00:08.904224 14041 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:08.904978 14041 heartbeater.cc:511] Master 127.13.121.254:39955 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
I20260927 07:00:08.907174 13851 ts_manager.cc:194] Registered new tserver with Master: d156aa15fefb4efba56b63167830416b (127.13.121.193:44977)
I20260927 07:00:08.908645 13851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:56843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:00:08.926960 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39955
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:45789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:09.248159 14047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:09.248461 14047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:09.248554 14047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:09.255815 14047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:09.255939 14047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:09.256052 14047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:09.265982 14047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:45789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39955
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:09.267293 14047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:09.268616 14047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:09.281165 14052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:09.281194 14053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:09.281291 14055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:09.282138 14047 server_base.cc:1034] running on GCE node
I20260927 07:00:09.282766 14047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:09.283748 14047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:09.284934 14047 hybrid_clock.cc:648] HybridClock initialized: now 1790492409284879 us; error 52 us; skew 500 ppm
I20260927 07:00:09.285218 14047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:09 dist-test-slave-p6hz krb5kdc[13806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:09.288224 14047 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:00:09.289598 14047 webserver.cc:460] Webserver started at http://127.13.121.194:42671/ using document root <none> and password file <none>
I20260927 07:00:09.290244 14047 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:09.290361 14047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:09.290647 14047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:09.293499 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "4ca7937776224c949aac3d058b1130a1"
format_stamp: "Formatted at 2026-09-27 07:00:09 on dist-test-slave-p6hz"
server_key: "7c17af92fa08084beb3cd16a137fd160"
server_key_iv: "aedea991783e48708a50fa760b295a17"
server_key_version: "encryptionkey@0"
I20260927 07:00:09.294215 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "4ca7937776224c949aac3d058b1130a1"
format_stamp: "Formatted at 2026-09-27 07:00:09 on dist-test-slave-p6hz"
server_key: "7c17af92fa08084beb3cd16a137fd160"
server_key_iv: "aedea991783e48708a50fa760b295a17"
server_key_version: "encryptionkey@0"
I20260927 07:00:09.299440 14047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260927 07:00:09.303231 14062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:09.304700 14047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:09.304860 14047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "4ca7937776224c949aac3d058b1130a1"
format_stamp: "Formatted at 2026-09-27 07:00:09 on dist-test-slave-p6hz"
server_key: "7c17af92fa08084beb3cd16a137fd160"
server_key_iv: "aedea991783e48708a50fa760b295a17"
server_key_version: "encryptionkey@0"
I20260927 07:00:09.305045 14047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:09.316823 14047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:09.320931 14047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:09.321183 14047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:09.322079 14047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:09.323427 14047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:09.323498 14047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:09.323611 14047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:09.323657 14047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:09.351444 14047 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:43349
I20260927 07:00:09.351478 14174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:43349 every 8 connection(s)
I20260927 07:00:09.352698 14047 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:09.362628 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 14047
I20260927 07:00:09.362845 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:09.363392 13799 external_mini_cluster.cc:1428] Setting key 563d85b8d0222261c116fb403955fb4a
Sep 27 07:00:09 dist-test-slave-p6hz krb5kdc[13806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:09.374274 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:09.355216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:39683 (local address 127.13.121.254:39955)
0927 07:00:09.355439 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:09.355446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:09.356802 (+ 1356us) server_negotiation.cc:407] Connection header received
0927 07:00:09.358629 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:09.358636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:09.358738 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:09.358845 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:09.360568 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.361356 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.363434 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.363886 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.363993 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:09.367844 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:09.367851 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:09.367855 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:09.367894 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:09.371064 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.371933 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.371937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.371939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.372077 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.372530 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.372534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.372536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.372804 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:09.372941 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:09.373681 (+ 740us) server_negotiation.cc:299] Negotiation successful
0927 07:00:09.373925 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260927 07:00:09.375358 14177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:09.355741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39955 (local address 127.13.121.194:39683)
0927 07:00:09.356482 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:09.356540 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:09.358341 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:09.359121 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:09.359131 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:09.359736 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:09.360381 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:09.360400 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.361537 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.361542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:09.363289 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:09.363303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.364118 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.364126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:09.364338 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:09.364965 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:09.364995 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:09.367504 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:09.371280 (+ 3776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:09.371291 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:09.371312 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:09.371793 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:09.372193 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:09.372197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:09.372201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:09.372406 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:09.372935 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:09.372943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:09.373305 (+ 362us) client_negotiation.cc:769] Sending connection context
0927 07:00:09.373557 (+ 252us) client_negotiation.cc:240] Negotiation successful
0927 07:00:09.374126 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":272,"threads_started":1}
I20260927 07:00:09.376899 14175 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39955
I20260927 07:00:09.377267 14175 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:00:09.378230 14175 heartbeater.cc:511] Master 127.13.121.254:39955 requested a full tablet report, sending...
I20260927 07:00:09.379757 13851 ts_manager.cc:194] Registered new tserver with Master: 4ca7937776224c949aac3d058b1130a1 (127.13.121.194:43349)
I20260927 07:00:09.380582 13851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:39683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:00:09.402595 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39955
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:45789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:09.724043 14180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:09.724361 14180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:09.724460 14180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:09.732684 14180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:09.732836 14180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:09.732954 14180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:09.743613 14180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:45789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39955
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:09.744936 14180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:09.746258 14180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:09.760169 14186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:09.760210 14185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:09.760294 14188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:09.760939 14180 server_base.cc:1034] running on GCE node
I20260927 07:00:09.761590 14180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:09.762665 14180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:09.763870 14180 hybrid_clock.cc:648] HybridClock initialized: now 1790492409763823 us; error 42 us; skew 500 ppm
I20260927 07:00:09.764194 14180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:09 dist-test-slave-p6hz krb5kdc[13806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:09.767313 14180 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:00:09.768774 14180 webserver.cc:460] Webserver started at http://127.13.121.195:33599/ using document root <none> and password file <none>
I20260927 07:00:09.769497 14180 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:09.769603 14180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:09.769878 14180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:09.772809 14180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "f9c43caac39f4b77a0a539ed46bd565e"
format_stamp: "Formatted at 2026-09-27 07:00:09 on dist-test-slave-p6hz"
server_key: "4bcb8c52b7b5bd6e2fdf2baeecd57d82"
server_key_iv: "f4037df06236b8a37c49f27cd33f1ca8"
server_key_version: "encryptionkey@0"
I20260927 07:00:09.773545 14180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "f9c43caac39f4b77a0a539ed46bd565e"
format_stamp: "Formatted at 2026-09-27 07:00:09 on dist-test-slave-p6hz"
server_key: "4bcb8c52b7b5bd6e2fdf2baeecd57d82"
server_key_iv: "f4037df06236b8a37c49f27cd33f1ca8"
server_key_version: "encryptionkey@0"
I20260927 07:00:09.779098 14180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:00:09.783247 14195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:09.784759 14180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:09.784919 14180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "f9c43caac39f4b77a0a539ed46bd565e"
format_stamp: "Formatted at 2026-09-27 07:00:09 on dist-test-slave-p6hz"
server_key: "4bcb8c52b7b5bd6e2fdf2baeecd57d82"
server_key_iv: "f4037df06236b8a37c49f27cd33f1ca8"
server_key_version: "encryptionkey@0"
I20260927 07:00:09.785101 14180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:09.797176 14180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:09.802764 14180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:09.803048 14180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:09.804404 14180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:09.806548 14180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:09.806641 14180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:09.806732 14180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:09.806783 14180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:09.838238 14180 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:37475
I20260927 07:00:09.838347 14307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:37475 every 8 connection(s)
I20260927 07:00:09.839648 14180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:09.848206 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 14180
I20260927 07:00:09.848380 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:09.848763 13799 external_mini_cluster.cc:1428] Setting key 61e1a6789d9f974405f50184c6ff57a8
Sep 27 07:00:09 dist-test-slave-p6hz krb5kdc[13806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:09.858623 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:09.842106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:38023 (local address 127.13.121.254:39955)
0927 07:00:09.842276 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:09.842281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:09.843529 (+ 1248us) server_negotiation.cc:407] Connection header received
0927 07:00:09.845349 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:09.845355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:09.845435 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:09.845522 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:09.847079 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.847837 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.849516 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.850054 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.850168 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:09.853610 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:09.853615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:09.853619 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:09.853652 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:09.856045 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.856780 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.856784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.856786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.856911 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.857237 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.857240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.857242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.857490 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:09.857590 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:09.858214 (+ 624us) server_negotiation.cc:299] Negotiation successful
0927 07:00:09.858381 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 07:00:09.859735 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:09.842629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39955 (local address 127.13.121.195:38023)
0927 07:00:09.843375 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:09.843419 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:09.845042 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:09.845786 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:09.845795 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:09.846327 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:09.846905 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:09.846924 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.847991 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.847995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:09.849367 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:09.849385 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.850292 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.850300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:09.850522 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:09.851183 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:09.851206 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:09.853426 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:09.856187 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:09.856195 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:09.856210 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:09.856684 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:09.856985 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:09.856988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:09.856990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:09.857163 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:09.857576 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:09.857581 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:09.857917 (+ 336us) client_negotiation.cc:769] Sending connection context
0927 07:00:09.858152 (+ 235us) client_negotiation.cc:240] Negotiation successful
0927 07:00:09.858571 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":256,"threads_started":1}
I20260927 07:00:09.861256 14308 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39955
I20260927 07:00:09.861585 14308 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:09.862335 14308 heartbeater.cc:511] Master 127.13.121.254:39955 requested a full tablet report, sending...
I20260927 07:00:09.863922 13851 ts_manager.cc:194] Registered new tserver with Master: f9c43caac39f4b77a0a539ed46bd565e (127.13.121.195:37475)
I20260927 07:00:09.864862 13851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:38023
I20260927 07:00:09.864892 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:00:09 dist-test-slave-p6hz krb5kdc[13806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:00:09 dist-test-slave-p6hz krb5kdc[13806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:09.908591 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:09.893638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.13.121.193:44977)
0927 07:00:09.894231 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:09.894238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:09.894268 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:00:09.894425 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:09.894432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:09.894648 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:09.894788 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:09.896162 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.897176 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.898418 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.898980 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.899077 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:09.903086 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:09.903103 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:09.903130 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:09.903229 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:09.905957 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.906683 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.906691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.906694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.906845 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.907189 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.907194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.907197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.907547 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:09.907766 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:09.907994 (+ 228us) server_negotiation.cc:299] Negotiation successful
0927 07:00:09.908201 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":302,"threads_started":1}
W20260927 07:00:09.909893 13956 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56724
I20260927 07:00:09.911983 14041 heartbeater.cc:503] Master 127.13.121.254:39955 was elected leader, sending a full tablet report...
Sep 27 07:00:09 dist-test-slave-p6hz krb5kdc[13806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 07:00:09 dist-test-slave-p6hz krb5kdc[13806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:09.950323 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:09.937158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.13.121.193:44977)
0927 07:00:09.937360 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:09.937368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:09.937705 (+ 337us) server_negotiation.cc:407] Connection header received
0927 07:00:09.937847 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:09.937851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:09.937943 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:09.938025 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:09.939366 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.940054 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.941196 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.941606 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.941716 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:09.945415 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:09.945420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:09.945424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:09.945459 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:09.947955 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.948569 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.948575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.948577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.948707 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.949051 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.949054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.949057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.949307 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:09.949516 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:09.949813 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 07:00:09.950023 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 07:00:09.951678 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 13913
I20260927 07:00:09.964509 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:09.952527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.13.121.193:44977)
0927 07:00:09.952718 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:09.952724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:09.952986 (+ 262us) server_negotiation.cc:407] Connection header received
0927 07:00:09.953090 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:09.953093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:09.953146 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:09.953227 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:09.954272 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.954958 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.956175 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:09.956594 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:09.956687 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:09.958936 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:09.958941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:09.958945 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:09.958978 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:09.961825 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.962596 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.962600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.962602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.962783 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:09.963173 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:09.963178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:09.963182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:09.963454 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:09.963679 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:09.963965 (+ 286us) server_negotiation.cc:299] Negotiation successful
0927 07:00:09.964169 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 07:00:09.965193 14036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:10.046953 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 13913
I20260927 07:00:10.059473 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 14047
Sep 27 07:00:10 dist-test-slave-p6hz krb5kdc[13806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:00:10.078215 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:10.060358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39908 (local address 127.13.121.194:43349)
0927 07:00:10.060882 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:10.060889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:10.060915 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:00:10.060971 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:10.060977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:10.061179 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:10.061304 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:10.062580 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:10.063937 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:10.065305 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:10.065882 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:10.065997 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:10.072673 (+ 6676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:10.072688 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:10.072703 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:10.072742 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:10.075723 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:10.076433 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:10.076440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:10.076442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:10.076573 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:10.076866 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:10.076869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:10.076871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:10.077223 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:10.077460 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:10.077726 (+ 266us) server_negotiation.cc:299] Negotiation successful
0927 07:00:10.077932 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260927 07:00:10.079486 14170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:10.165177 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 14047
I20260927 07:00:10.177336 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 14180
Sep 27 07:00:10 dist-test-slave-p6hz krb5kdc[13806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:00:10.196431 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:10.178138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.13.121.195:37475)
0927 07:00:10.178693 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:10.178700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:10.178729 (+ 29us) server_negotiation.cc:407] Connection header received
0927 07:00:10.178787 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:10.178793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:10.179002 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:10.179127 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:10.180419 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:10.181440 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:10.182835 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:10.183489 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:10.183605 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:10.188642 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:10.188683 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:10.188705 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:10.188761 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:10.193042 (+ 4281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:10.193945 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:10.193955 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:10.193958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:10.194109 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:10.194483 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:10.194488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:10.194492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:10.194973 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:10.195295 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:10.195675 (+ 380us) server_negotiation.cc:299] Negotiation successful
0927 07:00:10.195965 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":270,"threads_started":1}
I20260927 07:00:10.198220 14303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:10.285995 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 14180
I20260927 07:00:10.298225 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 13819
I20260927 07:00:10.299494 13881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:10.400393 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 13819
2026-09-27T07:00:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3697 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:10 dist-test-slave-p6hz krb5kdc[14329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:10 dist-test-slave-p6hz krb5kdc[14329](info): set up 2 sockets
Sep 27 07:00:10 dist-test-slave-p6hz krb5kdc[14329](info): commencing operation
krb5kdc: starting...
W20260927 07:00:11.391057 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.003s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:11 dist-test-slave-p6hz krb5kdc[14329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:11.567502 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:37775
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:37775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:11.884236 14342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:11.884553 14342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:11.884670 14342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:11.892035 14342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:11.892155 14342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:11.892227 14342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:11.892343 14342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:11.892396 14342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:11.902442 14342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:37775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:37775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:11.903851 14342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:11.905119 14342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:11.916689 14347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:11.916668 14350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:11.916630 14348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:11.917490 14342 server_base.cc:1034] running on GCE node
I20260927 07:00:11.918254 14342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:11.919888 14342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:11.921115 14342 hybrid_clock.cc:648] HybridClock initialized: now 1790492411921060 us; error 51 us; skew 500 ppm
I20260927 07:00:11.921514 14342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:11 dist-test-slave-p6hz krb5kdc[14329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492411, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:11.929909 14342 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:11.931629 14342 webserver.cc:460] Webserver started at http://127.13.121.254:42641/ using document root <none> and password file <none>
I20260927 07:00:11.932368 14342 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:11.932482 14342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:11.932811 14342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:11.935956 14342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "c71e6fdbbe9d4e169a09a5274d94465e"
format_stamp: "Formatted at 2026-09-27 07:00:11 on dist-test-slave-p6hz"
server_key: "028ebb3aaba169dd76500931b491d103"
server_key_iv: "817920e25b38538e755ce78c2dcca7f4"
server_key_version: "encryptionkey@0"
I20260927 07:00:11.936717 14342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "c71e6fdbbe9d4e169a09a5274d94465e"
format_stamp: "Formatted at 2026-09-27 07:00:11 on dist-test-slave-p6hz"
server_key: "028ebb3aaba169dd76500931b491d103"
server_key_iv: "817920e25b38538e755ce78c2dcca7f4"
server_key_version: "encryptionkey@0"
I20260927 07:00:11.942211 14342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260927 07:00:11.946167 14357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:11.947788 14342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260927 07:00:11.947984 14342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "c71e6fdbbe9d4e169a09a5274d94465e"
format_stamp: "Formatted at 2026-09-27 07:00:11 on dist-test-slave-p6hz"
server_key: "028ebb3aaba169dd76500931b491d103"
server_key_iv: "817920e25b38538e755ce78c2dcca7f4"
server_key_version: "encryptionkey@0"
I20260927 07:00:11.948186 14342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:11.987097 14342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:11.991446 14342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:11.991721 14342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:12.009382 14342 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:37775
I20260927 07:00:12.009423 14408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:37775 every 8 connection(s)
I20260927 07:00:12.010768 14342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:12.014340 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 14342
I20260927 07:00:12.014577 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:12.014941 14409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:12.015373 13799 external_mini_cluster.cc:1428] Setting key 28a49110818b43f75c7a231b9ebbfb29
I20260927 07:00:12.029946 14409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e: Bootstrap starting.
Sep 27 07:00:12 dist-test-slave-p6hz krb5kdc[14329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:12.033485 14409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:12.034600 14409 log.cc:824] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:12.037572 14409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e: No bootstrap required, opened a new log
I20260927 07:00:12.039826 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:12.017459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34088 (local address 127.13.121.254:37775)
0927 07:00:12.018339 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:12.018351 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:12.018398 (+ 47us) server_negotiation.cc:407] Connection header received
0927 07:00:12.020025 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:12.020051 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:12.020489 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:12.020848 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:12.022628 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:12.024022 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:12.025811 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:12.026578 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:12.026744 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:12.031674 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:12.031692 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:12.031712 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:12.031766 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:12.035281 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:12.036108 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:12.036120 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:12.036129 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:12.036402 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:12.036855 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:12.036859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:12.036861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:12.037426 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:12.037697 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:12.038265 (+ 568us) server_negotiation.cc:299] Negotiation successful
0927 07:00:12.038696 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":409,"threads_started":1}
I20260927 07:00:12.046108 14409 raft_consensus.cc:348] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71e6fdbbe9d4e169a09a5274d94465e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37775 } }
I20260927 07:00:12.046609 14409 raft_consensus.cc:374] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:12.046713 14409 raft_consensus.cc:729] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c71e6fdbbe9d4e169a09a5274d94465e, State: Initialized, Role: FOLLOWER
I20260927 07:00:12.047272 14409 consensus_queue.cc:260] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71e6fdbbe9d4e169a09a5274d94465e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37775 } }
I20260927 07:00:12.047485 14409 raft_consensus.cc:388] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:12.047581 14409 raft_consensus.cc:482] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:12.047698 14409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:12.049738 14409 raft_consensus.cc:504] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71e6fdbbe9d4e169a09a5274d94465e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37775 } }
I20260927 07:00:12.050233 14409 leader_election.cc:302] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c71e6fdbbe9d4e169a09a5274d94465e; no voters:
I20260927 07:00:12.050746 14409 leader_election.cc:288] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:12.050906 14414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:12.051924 14414 raft_consensus.cc:686] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [term 1 LEADER]: Becoming Leader. State: Replica: c71e6fdbbe9d4e169a09a5274d94465e, State: Running, Role: LEADER
I20260927 07:00:12.052520 14414 consensus_queue.cc:237] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71e6fdbbe9d4e169a09a5274d94465e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37775 } }
I20260927 07:00:12.053006 14409 sys_catalog.cc:564] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:12.054906 14415 sys_catalog.cc:455] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c71e6fdbbe9d4e169a09a5274d94465e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71e6fdbbe9d4e169a09a5274d94465e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37775 } } }
I20260927 07:00:12.055336 14415 sys_catalog.cc:458] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:12.058415 14416 sys_catalog.cc:455] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c71e6fdbbe9d4e169a09a5274d94465e. Latest consensus state: current_term: 1 leader_uuid: "c71e6fdbbe9d4e169a09a5274d94465e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71e6fdbbe9d4e169a09a5274d94465e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37775 } } }
I20260927 07:00:12.058763 14416 sys_catalog.cc:458] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:12.060678 14420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:12.066718 14420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:12.076385 14420 catalog_manager.cc:1347] Generated new cluster ID: 6eec0222d08b41ff90c670be0dcf0c1e
I20260927 07:00:12.076514 14420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:12.104074 14420 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:12.105963 14420 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:12.125479 14420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c71e6fdbbe9d4e169a09a5274d94465e: Generated new TSK 0
I20260927 07:00:12.126856 14420 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:00:12.178440 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:37775
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:12.499681 14435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:12.499977 14435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:12.500092 14435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:12.507426 14435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:12.507545 14435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:12.507696 14435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:12.517714 14435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:37775
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:12.519002 14435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:12.520375 14435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:12.533239 14441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:12.533253 14443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:12.533325 14440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:12.534468 14435 server_base.cc:1034] running on GCE node
I20260927 07:00:12.535111 14435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:12.536160 14435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:12.537415 14435 hybrid_clock.cc:648] HybridClock initialized: now 1790492412537358 us; error 47 us; skew 500 ppm
I20260927 07:00:12.537760 14435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:12 dist-test-slave-p6hz krb5kdc[14329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492412, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:12.541229 14435 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:00:12.542745 14435 webserver.cc:460] Webserver started at http://127.13.121.193:38727/ using document root <none> and password file <none>
I20260927 07:00:12.543524 14435 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:12.543641 14435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:12.543907 14435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:12.546825 14435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "015d597094034d3bb76f070e19793b32"
format_stamp: "Formatted at 2026-09-27 07:00:12 on dist-test-slave-p6hz"
server_key: "7c6fea157e0167c92b6090579bbce6da"
server_key_iv: "0048c12011cd0b3260324e017ba06910"
server_key_version: "encryptionkey@0"
I20260927 07:00:12.547655 14435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "015d597094034d3bb76f070e19793b32"
format_stamp: "Formatted at 2026-09-27 07:00:12 on dist-test-slave-p6hz"
server_key: "7c6fea157e0167c92b6090579bbce6da"
server_key_iv: "0048c12011cd0b3260324e017ba06910"
server_key_version: "encryptionkey@0"
I20260927 07:00:12.553143 14435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:00:12.556915 14450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:12.558401 14435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:00:12.558563 14435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "015d597094034d3bb76f070e19793b32"
format_stamp: "Formatted at 2026-09-27 07:00:12 on dist-test-slave-p6hz"
server_key: "7c6fea157e0167c92b6090579bbce6da"
server_key_iv: "0048c12011cd0b3260324e017ba06910"
server_key_version: "encryptionkey@0"
I20260927 07:00:12.558758 14435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:12.586747 14435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:12.590915 14435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:12.591216 14435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:12.592209 14435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:12.593523 14435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:12.593600 14435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:12.593704 14435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:12.593762 14435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:12.623353 14435 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:45207
I20260927 07:00:12.623395 14562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:45207 every 8 connection(s)
I20260927 07:00:12.624648 14435 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:12.634694 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 14435
I20260927 07:00:12.634904 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:12.635717 13799 external_mini_cluster.cc:1428] Setting key 5645c03f542b4de3014aba7db196ccf0
Sep 27 07:00:12 dist-test-slave-p6hz krb5kdc[14329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492412, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:12.644665 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:12.627178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:35859 (local address 127.13.121.254:37775)
0927 07:00:12.627579 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:12.627584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:12.628547 (+ 963us) server_negotiation.cc:407] Connection header received
0927 07:00:12.630335 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:12.630340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:12.630407 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:12.630507 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:12.632024 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:12.632781 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:12.634303 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:12.634706 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:12.634809 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:12.638953 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:12.638959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:12.638963 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:12.638998 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:12.641528 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:12.642275 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:12.642281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:12.642285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:12.642419 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:12.642900 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:12.642904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:12.642906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:12.643237 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:12.643347 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:12.644199 (+ 852us) server_negotiation.cc:299] Negotiation successful
0927 07:00:12.644369 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":206,"threads_started":1}
I20260927 07:00:12.645982 14566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:12.627685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:37775 (local address 127.13.121.193:35859)
0927 07:00:12.628338 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:12.628400 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:12.630065 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:12.630707 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:12.630715 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:12.631282 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:12.631850 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:12.631868 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:12.632928 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:12.632932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:12.634164 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:12.634175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:12.635014 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:12.635023 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:12.635348 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:12.636115 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:12.636139 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:12.638697 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:12.641688 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:12.641695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:12.641710 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:12.642147 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:12.642519 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:12.642524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:12.642527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:12.642786 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:12.643349 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:12.643358 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:12.643798 (+ 440us) client_negotiation.cc:769] Sending connection context
0927 07:00:12.644074 (+ 276us) client_negotiation.cc:240] Negotiation successful
0927 07:00:12.644612 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":248,"threads_started":1}
I20260927 07:00:12.647872 14563 heartbeater.cc:347] Connected to a master server at 127.13.121.254:37775
I20260927 07:00:12.648185 14563 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:12.648918 14563 heartbeater.cc:511] Master 127.13.121.254:37775 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:00:12.651328 14374 ts_manager.cc:194] Registered new tserver with Master: 015d597094034d3bb76f070e19793b32 (127.13.121.193:45207)
I20260927 07:00:12.652849 14374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:35859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:00:12.674674 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:37775
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:12.993990 14569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:12.994328 14569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:12.994450 14569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:13.001914 14569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:13.002059 14569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:13.002216 14569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:13.014444 14569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:37775
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:13.016141 14569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:13.017841 14569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:13.033375 14575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:13.033407 14574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:13.033914 14569 server_base.cc:1034] running on GCE node
W20260927 07:00:13.033478 14577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:13.034871 14569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:13.036005 14569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:13.037232 14569 hybrid_clock.cc:648] HybridClock initialized: now 1790492413037172 us; error 57 us; skew 500 ppm
I20260927 07:00:13.037580 14569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:13 dist-test-slave-p6hz krb5kdc[14329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:13.040908 14569 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:00:13.042412 14569 webserver.cc:460] Webserver started at http://127.13.121.194:42331/ using document root <none> and password file <none>
I20260927 07:00:13.043125 14569 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:13.043282 14569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:13.043560 14569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:13.046497 14569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "bcbedcdc41734cf9ba73c1396a5f34e9"
format_stamp: "Formatted at 2026-09-27 07:00:13 on dist-test-slave-p6hz"
server_key: "9c6ec49ec22cb048b5ba74a6e0ccf49b"
server_key_iv: "7e9bd7d500dce238364c40747b4dc9d5"
server_key_version: "encryptionkey@0"
I20260927 07:00:13.047312 14569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "bcbedcdc41734cf9ba73c1396a5f34e9"
format_stamp: "Formatted at 2026-09-27 07:00:13 on dist-test-slave-p6hz"
server_key: "9c6ec49ec22cb048b5ba74a6e0ccf49b"
server_key_iv: "7e9bd7d500dce238364c40747b4dc9d5"
server_key_version: "encryptionkey@0"
I20260927 07:00:13.052490 14569 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260927 07:00:13.056232 14584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:13.057723 14569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:00:13.057899 14569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "bcbedcdc41734cf9ba73c1396a5f34e9"
format_stamp: "Formatted at 2026-09-27 07:00:13 on dist-test-slave-p6hz"
server_key: "9c6ec49ec22cb048b5ba74a6e0ccf49b"
server_key_iv: "7e9bd7d500dce238364c40747b4dc9d5"
server_key_version: "encryptionkey@0"
I20260927 07:00:13.058086 14569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:13.073592 14569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:13.078119 14569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:13.078413 14569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:13.079433 14569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:13.080785 14569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:13.080865 14569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:13.080986 14569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:13.081041 14569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:13.113894 14696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:40523 every 8 connection(s)
I20260927 07:00:13.113996 14569 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:40523
I20260927 07:00:13.115921 14569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:13.120756 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 14569
I20260927 07:00:13.120952 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:13.121387 13799 external_mini_cluster.cc:1428] Setting key b644eeb4e8069a629f905e8ccae6deb1
Sep 27 07:00:13 dist-test-slave-p6hz krb5kdc[14329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
I20260927 07:00:13.141053 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.119717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:53815 (local address 127.13.121.254:37775)
0927 07:00:13.120013 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.120019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.121713 (+ 1694us) server_negotiation.cc:407] Connection header received
0927 07:00:13.124608 (+ 2895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.124614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.124729 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.124846 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:13.127199 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.127892 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.130027 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.130651 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.130782 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.134538 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:13.134546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:13.134552 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:13.134610 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:13.137494 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.138448 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.138453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.138455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.138609 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.139170 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.139176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.139180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.139549 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:13.139692 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.140451 (+ 759us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.140692 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260927 07:00:13.142138 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.120503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:37775 (local address 127.13.121.194:53815)
0927 07:00:13.121480 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:13.121536 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:13.123927 (+ 2391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:13.125288 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:13.125305 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:13.126054 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:13.126936 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.126966 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.128111 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.128130 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.129819 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.129834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.130898 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.130904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.131202 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.131912 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:13.131936 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:13.134235 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:13.137689 (+ 3454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:13.137697 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:13.137715 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:13.138292 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:13.138738 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:13.138743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:13.138745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:13.138997 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:13.139687 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:13.139696 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:13.140083 (+ 387us) client_negotiation.cc:769] Sending connection context
0927 07:00:13.140347 (+ 264us) client_negotiation.cc:240] Negotiation successful
0927 07:00:13.140920 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":389,"threads_started":1}
I20260927 07:00:13.143946 14697 heartbeater.cc:347] Connected to a master server at 127.13.121.254:37775
I20260927 07:00:13.144301 14697 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:13.145201 14697 heartbeater.cc:511] Master 127.13.121.254:37775 requested a full tablet report, sending...
I20260927 07:00:13.146917 14374 ts_manager.cc:194] Registered new tserver with Master: bcbedcdc41734cf9ba73c1396a5f34e9 (127.13.121.194:40523)
I20260927 07:00:13.147778 14374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:53815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:00:13.163381 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:37775
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:13.488497 14702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:13.488813 14702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:13.488953 14702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:13.496284 14702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:13.496423 14702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:13.496601 14702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:13.506611 14702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:37775
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:13.508008 14702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:13.509325 14702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:13.522524 14707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:13.522549 14708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:13.523289 14710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:13.523352 14702 server_base.cc:1034] running on GCE node
I20260927 07:00:13.523969 14702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:13.524922 14702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:13.526149 14702 hybrid_clock.cc:648] HybridClock initialized: now 1790492413526087 us; error 51 us; skew 500 ppm
I20260927 07:00:13.526472 14702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:13 dist-test-slave-p6hz krb5kdc[14329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:13.529760 14702 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:00:13.531422 14702 webserver.cc:460] Webserver started at http://127.13.121.195:43619/ using document root <none> and password file <none>
I20260927 07:00:13.532122 14702 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:13.532231 14702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:13.532536 14702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:13.535878 14702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "3b5f7257cb584d18a85e56657ff9b63d"
format_stamp: "Formatted at 2026-09-27 07:00:13 on dist-test-slave-p6hz"
server_key: "0afb132fcc2a2990b1e82a1f833e2072"
server_key_iv: "d5ffecfd5247f5500901c12cd96d237a"
server_key_version: "encryptionkey@0"
I20260927 07:00:13.536736 14702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "3b5f7257cb584d18a85e56657ff9b63d"
format_stamp: "Formatted at 2026-09-27 07:00:13 on dist-test-slave-p6hz"
server_key: "0afb132fcc2a2990b1e82a1f833e2072"
server_key_iv: "d5ffecfd5247f5500901c12cd96d237a"
server_key_version: "encryptionkey@0"
I20260927 07:00:13.542724 14702 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260927 07:00:13.546468 14717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:13.547992 14702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 07:00:13.548144 14702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "3b5f7257cb584d18a85e56657ff9b63d"
format_stamp: "Formatted at 2026-09-27 07:00:13 on dist-test-slave-p6hz"
server_key: "0afb132fcc2a2990b1e82a1f833e2072"
server_key_iv: "d5ffecfd5247f5500901c12cd96d237a"
server_key_version: "encryptionkey@0"
I20260927 07:00:13.548374 14702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:13.568087 14702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:13.572459 14702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:13.572841 14702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:13.573877 14702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:13.575260 14702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:13.575335 14702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:13.575443 14702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:13.575490 14702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:13.605448 14702 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:40255
I20260927 07:00:13.605490 14829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:40255 every 8 connection(s)
I20260927 07:00:13.606707 14702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:13.610108 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 14702
I20260927 07:00:13.610287 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:13.610968 13799 external_mini_cluster.cc:1428] Setting key 20d13905e60003ba9bc20035a9140a58
Sep 27 07:00:13 dist-test-slave-p6hz krb5kdc[14329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:13.627355 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.609199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:48615 (local address 127.13.121.254:37775)
0927 07:00:13.609421 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.609428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.610754 (+ 1326us) server_negotiation.cc:407] Connection header received
0927 07:00:13.612769 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.612773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.612842 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.612942 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:13.614779 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.615554 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.617026 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.617454 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.617561 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.621024 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:13.621030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:13.621033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:13.621071 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:13.624097 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.624936 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.624940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.624942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.625080 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.625572 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.625577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.625580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.625945 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:13.626344 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.626872 (+ 528us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.627043 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 07:00:13.628737 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.609701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:37775 (local address 127.13.121.195:48615)
0927 07:00:13.610519 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:13.610566 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:13.612504 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:13.613197 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:13.613210 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:13.613975 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:13.614593 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.614612 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.615674 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.615678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.616907 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.616918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.617684 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.617689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.617908 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.618569 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:13.618598 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:13.620804 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:13.624265 (+ 3461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:13.624273 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:13.624287 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:13.624779 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:13.625192 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:13.625198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:13.625202 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:13.625451 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:13.626076 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:13.626083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:13.626496 (+ 413us) client_negotiation.cc:769] Sending connection context
0927 07:00:13.626817 (+ 321us) client_negotiation.cc:240] Negotiation successful
0927 07:00:13.627389 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":309,"threads_started":1}
I20260927 07:00:13.630043 14830 heartbeater.cc:347] Connected to a master server at 127.13.121.254:37775
I20260927 07:00:13.630318 14830 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:13.631006 14830 heartbeater.cc:511] Master 127.13.121.254:37775 requested a full tablet report, sending...
I20260927 07:00:13.632414 14374 ts_manager.cc:194] Registered new tserver with Master: 3b5f7257cb584d18a85e56657ff9b63d (127.13.121.195:40255)
I20260927 07:00:13.633186 14374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:48615
I20260927 07:00:13.638628 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:00:13.656662 14563 heartbeater.cc:503] Master 127.13.121.254:37775 was elected leader, sending a full tablet report...
I20260927 07:00:13.656775 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.644718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.13.121.254:37775)
0927 07:00:13.644930 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.644937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.645263 (+ 326us) server_negotiation.cc:407] Connection header received
0927 07:00:13.645396 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.645400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.645462 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.645537 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:13.646876 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.647661 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.648895 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.649271 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.649355 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.651503 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:13.651508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:13.651511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:13.651547 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:13.654355 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.655010 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.655015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.655018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.655251 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.655594 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.655598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.655600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.655912 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:13.656098 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.656388 (+ 290us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.656547 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:00:13.663522 14373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:00:13.670388 14373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:13.692855 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.682747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.13.121.193:45207)
0927 07:00:13.685379 (+ 2632us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.685387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.685411 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:00:13.685495 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.685502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.685724 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.686563 (+ 839us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:13.686814 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.688210 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.690329 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.691714 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.691913 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.692057 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.692358 (+ 301us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.692534 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2462,"thread_start_us":2332,"threads_started":1}
I20260927 07:00:13.692940 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.682455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45207 (local address 127.0.0.1:56244)
0927 07:00:13.683337 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:13.683382 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:13.683543 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:13.685928 (+ 2385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:13.685936 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:13.685957 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:13.686411 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.686421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.688325 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.688328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.690173 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.690188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.691912 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.691918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.692105 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.692161 (+ 56us) client_negotiation.cc:769] Sending connection context
0927 07:00:13.692299 (+ 138us) client_negotiation.cc:240] Negotiation successful
0927 07:00:13.692493 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":52,"thread_start_us":1876,"threads_started":1}
I20260927 07:00:13.693454 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.683376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.13.121.194:40523)
0927 07:00:13.684206 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.684213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.684840 (+ 627us) server_negotiation.cc:407] Connection header received
0927 07:00:13.684940 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.684948 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.685196 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.685372 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:13.686300 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.687869 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.690212 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.691714 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.691880 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.692057 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.692957 (+ 900us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.693122 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"spinlock_wait_cycles":17792,"thread_start_us":663,"threads_started":1}
I20260927 07:00:13.693704 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.682800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:40523 (local address 127.0.0.1:33140)
0927 07:00:13.684588 (+ 1788us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:13.684605 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:13.684712 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:13.685620 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:13.685627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:13.685653 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:13.686127 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.686139 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.688017 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.688022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.690060 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.690075 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.692189 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.692193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.692332 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.692362 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 07:00:13.692442 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 07:00:13.692562 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1661,"mutex_wait_us":32,"thread_start_us":1519,"threads_started":1}
I20260927 07:00:13.694005 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.683002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:40255 (local address 127.0.0.1:50238)
0927 07:00:13.686215 (+ 3213us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:13.686232 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:13.686363 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:13.687171 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:13.687175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:13.687199 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:13.687600 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.687611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.689454 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.689459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.691379 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.691393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.693333 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.693339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.693498 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.693529 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 07:00:13.693604 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 07:00:13.693730 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3077,"thread_start_us":2898,"threads_started":1}
I20260927 07:00:13.696931 14632 tablet_service.cc:1467] Processing CreateTablet for tablet 912b37fced70453a8d45b579f813dd32 (DEFAULT_TABLE table=test-table [id=6e7214bcdbe8480fbaa5cae20a697bbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:13.696933 14498 tablet_service.cc:1467] Processing CreateTablet for tablet 912b37fced70453a8d45b579f813dd32 (DEFAULT_TABLE table=test-table [id=6e7214bcdbe8480fbaa5cae20a697bbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:13.698117 14498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 912b37fced70453a8d45b579f813dd32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:13.698114 14632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 912b37fced70453a8d45b579f813dd32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:13.699437 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.683575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.13.121.195:40255)
0927 07:00:13.685730 (+ 2155us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.685739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.686335 (+ 596us) server_negotiation.cc:407] Connection header received
0927 07:00:13.686534 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.686541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.686699 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.686881 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:13.687793 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.689313 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.691551 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.692844 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.693020 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.693196 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.694016 (+ 820us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.694182 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1988,"thread_start_us":1842,"threads_started":1}
I20260927 07:00:13.703430 14765 tablet_service.cc:1467] Processing CreateTablet for tablet 912b37fced70453a8d45b579f813dd32 (DEFAULT_TABLE table=test-table [id=6e7214bcdbe8480fbaa5cae20a697bbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:13.705250 14765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 912b37fced70453a8d45b579f813dd32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:13.709375 14848 tablet_bootstrap.cc:492] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32: Bootstrap starting.
I20260927 07:00:13.712550 14849 tablet_bootstrap.cc:492] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9: Bootstrap starting.
I20260927 07:00:13.712821 14848 tablet_bootstrap.cc:654] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:13.714767 14848 log.cc:824] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:13.716013 14849 tablet_bootstrap.cc:654] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:13.717219 14849 log.cc:824] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:13.718111 14848 tablet_bootstrap.cc:492] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32: No bootstrap required, opened a new log
I20260927 07:00:13.718372 14848 ts_tablet_manager.cc:1397] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260927 07:00:13.719389 14852 tablet_bootstrap.cc:492] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d: Bootstrap starting.
I20260927 07:00:13.720548 14849 tablet_bootstrap.cc:492] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9: No bootstrap required, opened a new log
I20260927 07:00:13.720834 14849 ts_tablet_manager.cc:1397] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260927 07:00:13.722563 14852 tablet_bootstrap.cc:654] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:13.723865 14852 log.cc:824] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:13.726469 14852 tablet_bootstrap.cc:492] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d: No bootstrap required, opened a new log
I20260927 07:00:13.726724 14852 ts_tablet_manager.cc:1397] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260927 07:00:13.728662 14848 raft_consensus.cc:348] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } }
I20260927 07:00:13.729116 14848 raft_consensus.cc:374] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:13.729259 14848 raft_consensus.cc:729] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015d597094034d3bb76f070e19793b32, State: Initialized, Role: FOLLOWER
I20260927 07:00:13.730052 14848 consensus_queue.cc:260] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } }
I20260927 07:00:13.730851 14849 raft_consensus.cc:348] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } }
I20260927 07:00:13.731367 14849 raft_consensus.cc:374] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:13.731503 14849 raft_consensus.cc:729] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbedcdc41734cf9ba73c1396a5f34e9, State: Initialized, Role: FOLLOWER
I20260927 07:00:13.731922 14848 ts_tablet_manager.cc:1428] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
I20260927 07:00:13.732282 14849 consensus_queue.cc:260] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } }
I20260927 07:00:13.733395 14697 heartbeater.cc:503] Master 127.13.121.254:37775 was elected leader, sending a full tablet report...
I20260927 07:00:13.734012 14849 ts_tablet_manager.cc:1428] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260927 07:00:13.735772 14852 raft_consensus.cc:348] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } }
I20260927 07:00:13.736254 14852 raft_consensus.cc:374] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:13.736366 14852 raft_consensus.cc:729] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5f7257cb584d18a85e56657ff9b63d, State: Initialized, Role: FOLLOWER
I20260927 07:00:13.737010 14852 consensus_queue.cc:260] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } }
I20260927 07:00:13.738166 14830 heartbeater.cc:503] Master 127.13.121.254:37775 was elected leader, sending a full tablet report...
I20260927 07:00:13.738714 14852 ts_tablet_manager.cc:1428] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260927 07:00:13.742170 14856 raft_consensus.cc:482] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:13.742367 14856 raft_consensus.cc:504] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } }
I20260927 07:00:13.743916 14856 leader_election.cc:288] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015d597094034d3bb76f070e19793b32 (127.13.121.193:45207), bcbedcdc41734cf9ba73c1396a5f34e9 (127.13.121.194:40523)
I20260927 07:00:13.750136 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.744192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:40523 (local address 127.13.121.195:42967)
0927 07:00:13.744414 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:13.744433 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:13.744547 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:13.745337 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:13.745342 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:13.745369 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:13.745559 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.745567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.746606 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.746612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.748291 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.748304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.749428 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.749434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.749634 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.749677 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 07:00:13.749772 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 07:00:13.749873 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260927 07:00:13.750168 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.744896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:42967 (local address 127.13.121.194:40523)
0927 07:00:13.745070 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.745076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.745093 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:00:13.745156 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.745161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.745224 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.745291 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:13.745699 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.746449 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.748435 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.749285 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.749395 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.749445 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.749799 (+ 354us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.749929 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 07:00:13.750344 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.744499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:44429 (local address 127.13.121.193:45207)
0927 07:00:13.744781 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.744785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.745070 (+ 285us) server_negotiation.cc:407] Connection header received
0927 07:00:13.745199 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.745203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.745258 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.745357 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:13.745930 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.746753 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.748619 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.749499 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.749660 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.749724 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.749993 (+ 269us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.750127 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260927 07:00:13.750496 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.744238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45207 (local address 127.13.121.195:44429)
0927 07:00:13.744960 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:13.744977 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:13.745095 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:13.745381 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:13.745385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:13.745423 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:13.745754 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.745764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.746889 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.746892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.748487 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:13.748499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.749680 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.749684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:13.749842 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.749873 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 07:00:13.749957 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 07:00:13.750082 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":52,"thread_start_us":350,"threads_started":1}
I20260927 07:00:13.751279 14652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912b37fced70453a8d45b579f813dd32" candidate_uuid: "3b5f7257cb584d18a85e56657ff9b63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" is_pre_election: true
I20260927 07:00:13.751456 14518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912b37fced70453a8d45b579f813dd32" candidate_uuid: "3b5f7257cb584d18a85e56657ff9b63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d597094034d3bb76f070e19793b32" is_pre_election: true
I20260927 07:00:13.751780 14652 raft_consensus.cc:2454] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b5f7257cb584d18a85e56657ff9b63d in term 0.
I20260927 07:00:13.751971 14518 raft_consensus.cc:2454] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b5f7257cb584d18a85e56657ff9b63d in term 0.
I20260927 07:00:13.752475 14720 leader_election.cc:302] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b5f7257cb584d18a85e56657ff9b63d, bcbedcdc41734cf9ba73c1396a5f34e9; no voters:
I20260927 07:00:13.752887 14856 raft_consensus.cc:2781] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:13.752996 14856 raft_consensus.cc:482] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:13.753062 14856 raft_consensus.cc:3037] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:13.755277 14856 raft_consensus.cc:504] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } }
I20260927 07:00:13.755869 14856 leader_election.cc:288] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [CANDIDATE]: Term 1 election: Requested vote from peers 015d597094034d3bb76f070e19793b32 (127.13.121.193:45207), bcbedcdc41734cf9ba73c1396a5f34e9 (127.13.121.194:40523)
I20260927 07:00:13.756336 14518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912b37fced70453a8d45b579f813dd32" candidate_uuid: "3b5f7257cb584d18a85e56657ff9b63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015d597094034d3bb76f070e19793b32"
I20260927 07:00:13.756363 14652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912b37fced70453a8d45b579f813dd32" candidate_uuid: "3b5f7257cb584d18a85e56657ff9b63d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9"
I20260927 07:00:13.756541 14652 raft_consensus.cc:3037] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:13.756541 14518 raft_consensus.cc:3037] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:13.759639 14652 raft_consensus.cc:2454] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b5f7257cb584d18a85e56657ff9b63d in term 1.
I20260927 07:00:13.759639 14518 raft_consensus.cc:2454] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b5f7257cb584d18a85e56657ff9b63d in term 1.
I20260927 07:00:13.760085 14720 leader_election.cc:302] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b5f7257cb584d18a85e56657ff9b63d, bcbedcdc41734cf9ba73c1396a5f34e9; no voters:
I20260927 07:00:13.760365 14856 raft_consensus.cc:2781] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:13.760819 14856 raft_consensus.cc:686] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [term 1 LEADER]: Becoming Leader. State: Replica: 3b5f7257cb584d18a85e56657ff9b63d, State: Running, Role: LEADER
I20260927 07:00:13.761322 14856 consensus_queue.cc:237] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } }
I20260927 07:00:13.766506 14372 catalog_manager.cc:5495] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b5f7257cb584d18a85e56657ff9b63d (127.13.121.195). New cstate: current_term: 1 leader_uuid: "3b5f7257cb584d18a85e56657ff9b63d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015d597094034d3bb76f070e19793b32" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcbedcdc41734cf9ba73c1396a5f34e9" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b5f7257cb584d18a85e56657ff9b63d" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40255 } health_report { overall_health: HEALTHY } } }
Sep 27 07:00:13 dist-test-slave-p6hz krb5kdc[14329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:13.800901 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.786712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.13.121.193:45207)
0927 07:00:13.786916 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.786923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.787423 (+ 500us) server_negotiation.cc:407] Connection header received
0927 07:00:13.787622 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.787627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.787691 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.787781 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:13.789306 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.790057 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.791335 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.791770 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.791878 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.795616 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:13.795632 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:13.795648 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:13.795686 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:13.798252 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.798897 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.798905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.798907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.799037 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.799409 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.799413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.799415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.799784 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:13.800049 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.800358 (+ 309us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.800566 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Sep 27 07:00:13 dist-test-slave-p6hz krb5kdc[14329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:00:13 dist-test-slave-p6hz krb5kdc[14329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:13.841370 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.828522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.13.121.193:45207)
0927 07:00:13.828729 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.828740 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.828896 (+ 156us) server_negotiation.cc:407] Connection header received
0927 07:00:13.829036 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.829040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.829105 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.829190 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:13.830434 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.831219 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.832435 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.832859 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.832963 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.836657 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:13.836663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:13.836667 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:13.836706 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:13.839233 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.839795 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.839799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.839801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.839936 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.840263 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.840265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.840268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.840518 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:13.840692 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.840904 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.841072 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260927 07:00:13.858256 14831 tablet.cc:2369] T 912b37fced70453a8d45b579f813dd32 P 3b5f7257cb584d18a85e56657ff9b63d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 07:00:13 dist-test-slave-p6hz krb5kdc[14329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260927 07:00:13.868561 14698 tablet.cc:2369] T 912b37fced70453a8d45b579f813dd32 P bcbedcdc41734cf9ba73c1396a5f34e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 07:00:13 dist-test-slave-p6hz krb5kdc[14329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
W20260927 07:00:13.877036 14564 tablet.cc:2369] T 912b37fced70453a8d45b579f813dd32 P 015d597094034d3bb76f070e19793b32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:00:13.881747 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.868624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56258 (local address 127.13.121.193:45207)
0927 07:00:13.868788 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.868795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.869179 (+ 384us) server_negotiation.cc:407] Connection header received
0927 07:00:13.869359 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.869363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.869426 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.869506 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:13.870789 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.871536 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.872717 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.873129 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.873236 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.876965 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:13.876974 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:13.876978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:13.877032 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:13.879661 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.880232 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.880235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.880237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.880354 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.880613 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.880616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.880617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.880885 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:13.881054 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.881263 (+ 209us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.881434 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260927 07:00:13.883435 14478 tablet_service.cc:2908] Rejecting scan request for tablet 912b37fced70453a8d45b579f813dd32: Uninitialized: clean time has not yet been initialized
I20260927 07:00:13.886947 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 14435
I20260927 07:00:13.899087 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:13.887714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56268 (local address 127.13.121.193:45207)
0927 07:00:13.887858 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:13.887861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:13.888177 (+ 316us) server_negotiation.cc:407] Connection header received
0927 07:00:13.888286 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:13.888290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:13.888362 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:13.888440 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:13.889640 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.890326 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.891582 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:13.892017 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:13.892169 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:13.894315 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:13.894320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:13.894324 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:13.894355 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:13.896779 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.897408 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.897412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.897414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.897535 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:13.897878 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:13.897882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:13.897884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:13.898167 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:13.898370 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:13.898663 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 07:00:13.898820 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 07:00:13.899780 14558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:13.989951 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 14435
I20260927 07:00:14.003284 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 14569
Sep 27 07:00:14 dist-test-slave-p6hz krb5kdc[14329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:00:14.019009 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:14.004201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.13.121.194:40523)
0927 07:00:14.004363 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:14.004367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:14.004384 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:00:14.004438 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:14.004442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:14.004500 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:14.004583 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:14.005820 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:14.006568 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:14.008027 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:14.008434 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:14.008546 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:14.012590 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:14.012610 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:14.012630 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:14.012677 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:14.016173 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:14.016874 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:14.016883 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:14.016887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:14.017039 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:14.017371 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:14.017376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:14.017379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:14.017862 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:14.018147 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:14.018382 (+ 235us) server_negotiation.cc:299] Negotiation successful
0927 07:00:14.018605 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 07:00:14.019790 14692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:14.106165 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 14569
I20260927 07:00:14.119572 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 14702
Sep 27 07:00:14 dist-test-slave-p6hz krb5kdc[14329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:00:14.135396 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:14.120328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.13.121.195:40255)
0927 07:00:14.120512 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:14.120517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:14.120536 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:00:14.120760 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:14.120765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:14.120850 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:14.120941 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:14.122227 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:14.123195 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:14.124778 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:14.125248 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:14.125365 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:14.129541 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:14.129558 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:14.129602 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:14.129645 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:14.132478 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:14.133291 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:14.133300 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:14.133302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:14.133458 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:14.133857 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:14.133862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:14.133864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:14.134256 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:14.134507 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:14.134852 (+ 345us) server_negotiation.cc:299] Negotiation successful
0927 07:00:14.135057 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:00:14.136121 14825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:14.223238 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 14702
I20260927 07:00:14.235926 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 14342
I20260927 07:00:14.237069 14404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:14.341746 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 14342
2026-09-27T07:00:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3942 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:14 dist-test-slave-p6hz krb5kdc[14877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:14 dist-test-slave-p6hz krb5kdc[14877](info): set up 2 sockets
Sep 27 07:00:14 dist-test-slave-p6hz krb5kdc[14877](info): commencing operation
krb5kdc: starting...
W20260927 07:00:15.343181 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.003s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:15 dist-test-slave-p6hz krb5kdc[14877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:15.518368 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41319
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:34659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:15.829173 14890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:15.829494 14890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:15.829635 14890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:15.837381 14890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:15.837520 14890 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 07:00:15.837621 14890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:15.837710 14890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:15.837771 14890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:15.837852 14890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:15.848249 14890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:34659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41319
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:15.849525 14890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:15.850816 14890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:15.862046 14898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:15.862150 14895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:15.862074 14896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:15.862728 14890 server_base.cc:1034] running on GCE node
I20260927 07:00:15.863574 14890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:15.864966 14890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:15.866199 14890 hybrid_clock.cc:648] HybridClock initialized: now 1790492415866131 us; error 45 us; skew 500 ppm
I20260927 07:00:15.866523 14890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:15 dist-test-slave-p6hz krb5kdc[14877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492415, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:15.869863 14890 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:15.871359 14890 webserver.cc:460] Webserver started at http://127.13.121.254:44233/ using document root <none> and password file <none>
I20260927 07:00:15.872062 14890 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:15.872159 14890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:15.872459 14890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:15.875474 14890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "6b85b1b44f964913a616874a0cc49381"
format_stamp: "Formatted at 2026-09-27 07:00:15 on dist-test-slave-p6hz"
server_key: "30d7ef9e4695d9127029479fbdc8ff4a"
server_key_iv: "a3ca7fce8bfdeb26c5ab737be4591eba"
server_key_version: "encryptionkey@0"
I20260927 07:00:15.876277 14890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "6b85b1b44f964913a616874a0cc49381"
format_stamp: "Formatted at 2026-09-27 07:00:15 on dist-test-slave-p6hz"
server_key: "30d7ef9e4695d9127029479fbdc8ff4a"
server_key_iv: "a3ca7fce8bfdeb26c5ab737be4591eba"
server_key_version: "encryptionkey@0"
I20260927 07:00:15.881654 14890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:00:15.885380 14905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:15.886955 14890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:00:15.887199 14890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "6b85b1b44f964913a616874a0cc49381"
format_stamp: "Formatted at 2026-09-27 07:00:15 on dist-test-slave-p6hz"
server_key: "30d7ef9e4695d9127029479fbdc8ff4a"
server_key_iv: "a3ca7fce8bfdeb26c5ab737be4591eba"
server_key_version: "encryptionkey@0"
I20260927 07:00:15.887429 14890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:15.902315 14890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:15.906432 14890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:15.906688 14890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:15.927704 14890 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:41319
I20260927 07:00:15.927709 14966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:41319 every 8 connection(s)
I20260927 07:00:15.929085 14890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:15.933029 14967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:15.934473 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 14890
I20260927 07:00:15.934711 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:15.935529 13799 external_mini_cluster.cc:1428] Setting key 1afdc5b46cbff3385a036db597e2d560
I20260927 07:00:15.947352 14967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381: Bootstrap starting.
Sep 27 07:00:15 dist-test-slave-p6hz krb5kdc[14877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:15.951751 14967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:15.952941 14967 log.cc:824] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:15.956163 14967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381: No bootstrap required, opened a new log
I20260927 07:00:15.959702 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:15.937640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37048 (local address 127.13.121.254:41319)
0927 07:00:15.938516 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:15.938531 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:15.938573 (+ 42us) server_negotiation.cc:407] Connection header received
0927 07:00:15.940248 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:15.940278 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:15.940713 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:15.941066 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:15.942791 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:15.944235 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:15.946082 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:15.946822 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:15.946968 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:15.951805 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:15.951813 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:15.951830 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:15.951871 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:15.955092 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:15.955998 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:15.956006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:15.956014 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:15.956252 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:15.956699 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:15.956706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:15.956709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:15.957293 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:15.957522 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:15.958048 (+ 526us) server_negotiation.cc:299] Negotiation successful
0927 07:00:15.958523 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":440,"threads_started":1}
I20260927 07:00:15.964215 14967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b85b1b44f964913a616874a0cc49381" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41319 } }
I20260927 07:00:15.964629 14967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:15.964730 14967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b85b1b44f964913a616874a0cc49381, State: Initialized, Role: FOLLOWER
I20260927 07:00:15.965368 14967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b85b1b44f964913a616874a0cc49381" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41319 } }
I20260927 07:00:15.965595 14967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:15.965718 14967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:15.965862 14967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:15.968075 14967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b85b1b44f964913a616874a0cc49381" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41319 } }
I20260927 07:00:15.968641 14967 leader_election.cc:302] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b85b1b44f964913a616874a0cc49381; no voters:
I20260927 07:00:15.969364 14967 leader_election.cc:288] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:15.969480 14972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:15.970160 14972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [term 1 LEADER]: Becoming Leader. State: Replica: 6b85b1b44f964913a616874a0cc49381, State: Running, Role: LEADER
I20260927 07:00:15.970669 14972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b85b1b44f964913a616874a0cc49381" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41319 } }
I20260927 07:00:15.971477 14967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:15.973835 14973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b85b1b44f964913a616874a0cc49381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b85b1b44f964913a616874a0cc49381" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41319 } } }
I20260927 07:00:15.974272 14973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:15.974954 14974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b85b1b44f964913a616874a0cc49381. Latest consensus state: current_term: 1 leader_uuid: "6b85b1b44f964913a616874a0cc49381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b85b1b44f964913a616874a0cc49381" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41319 } } }
I20260927 07:00:15.975286 14974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:15.976285 14978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:15.981446 14978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:15.991343 14978 catalog_manager.cc:1347] Generated new cluster ID: 18c8f6f599e44cebbd05d3874d62f62b
I20260927 07:00:15.991487 14978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:16.004376 14978 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:16.006258 14978 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:16.015239 14978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381: Generated new TSK 0
I20260927 07:00:16.016467 14978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:00:16.067706 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41319
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:34659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:16.393951 14993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:16.394248 14993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:16.394369 14993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:16.401661 14993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:16.401777 14993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:16.401882 14993 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:00:16.401983 14993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:16.412540 14993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:34659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.121.254:41319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:16.413955 14993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:16.415369 14993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:16.428428 14999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:16.428427 15001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:16.428450 14998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:16.429224 14993 server_base.cc:1034] running on GCE node
I20260927 07:00:16.429868 14993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:16.430851 14993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:16.432125 14993 hybrid_clock.cc:648] HybridClock initialized: now 1790492416431999 us; error 74 us; skew 500 ppm
I20260927 07:00:16.432448 14993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:16 dist-test-slave-p6hz krb5kdc[14877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492416, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:16.435649 14993 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:00:16.437215 14993 webserver.cc:460] Webserver started at http://127.13.121.193:39699/ using document root <none> and password file <none>
I20260927 07:00:16.437937 14993 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:16.438041 14993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:16.438385 14993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:16.441466 14993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "d90077940d6847a8958c0c81371c6cf8"
format_stamp: "Formatted at 2026-09-27 07:00:16 on dist-test-slave-p6hz"
server_key: "60d40fe63904152ffbea36e4d05a7c64"
server_key_iv: "3716507a11eadde90a0dae5c1c618b6d"
server_key_version: "encryptionkey@0"
I20260927 07:00:16.442344 14993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "d90077940d6847a8958c0c81371c6cf8"
format_stamp: "Formatted at 2026-09-27 07:00:16 on dist-test-slave-p6hz"
server_key: "60d40fe63904152ffbea36e4d05a7c64"
server_key_iv: "3716507a11eadde90a0dae5c1c618b6d"
server_key_version: "encryptionkey@0"
I20260927 07:00:16.447870 14993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 07:00:16.451622 15008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:16.453045 14993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:00:16.453243 14993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "d90077940d6847a8958c0c81371c6cf8"
format_stamp: "Formatted at 2026-09-27 07:00:16 on dist-test-slave-p6hz"
server_key: "60d40fe63904152ffbea36e4d05a7c64"
server_key_iv: "3716507a11eadde90a0dae5c1c618b6d"
server_key_version: "encryptionkey@0"
I20260927 07:00:16.453464 14993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:16.483764 14993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:16.488442 14993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:16.488720 14993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:16.491549 14993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:16.491642 14993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:16.491770 14993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:16.491824 14993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:00:16 dist-test-slave-p6hz krb5kdc[14877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492416, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:16.522058 15018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:16.493100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41319 (local address 127.13.121.193:41955)
0927 07:00:16.494064 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:16.494118 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:16.496734 (+ 2616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:16.502050 (+ 5316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:16.502060 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:16.502861 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:16.503936 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:16.503966 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.505511 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:16.505515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:16.506881 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:16.506896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.507961 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:16.507967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:16.508197 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:16.509097 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:16.509137 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:16.512690 (+ 3553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:16.516737 (+ 4047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:16.516745 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:16.516764 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:16.517391 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:16.517849 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:16.517857 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:16.517861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:16.518280 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:16.519017 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:16.519027 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:16.519604 (+ 577us) client_negotiation.cc:769] Sending connection context
0927 07:00:16.519908 (+ 304us) client_negotiation.cc:240] Negotiation successful
0927 07:00:16.520506 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":418,"threads_started":1}
I20260927 07:00:16.523113 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:16.492292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:41955 (local address 127.13.121.254:41319)
0927 07:00:16.501434 (+ 9142us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:16.501441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:16.501471 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:00:16.501563 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:16.501569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:16.501662 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:16.501812 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:16.504191 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.505348 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:16.507028 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.507687 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:16.507825 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:16.512965 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:16.512971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:16.512974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:16.513011 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:16.516556 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:16.517537 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:16.517541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:16.517543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:16.517700 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:16.518436 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:16.518454 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:16.518458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:16.518765 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:16.518908 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:16.522648 (+ 3740us) server_negotiation.cc:299] Negotiation successful
0927 07:00:16.522859 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8988,"thread_start_us":8892,"threads_started":1}
I20260927 07:00:16.542007 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:16.529220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:44605 (local address 127.13.121.254:41319)
0927 07:00:16.529448 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:16.529455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:16.529623 (+ 168us) server_negotiation.cc:407] Connection header received
0927 07:00:16.529807 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:16.529812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:16.529885 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:16.529967 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:16.530899 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.531766 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:16.533706 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.534117 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:16.534234 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:16.536518 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:16.536523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:16.536526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:16.536558 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:16.538940 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:16.539767 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:16.539773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:16.539776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:16.539925 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:16.540357 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:16.540362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:16.540367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:16.540752 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:16.540924 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:16.541432 (+ 508us) server_negotiation.cc:299] Negotiation successful
0927 07:00:16.541664 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 07:00:16.542111 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:16.528810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41319 (local address 127.13.121.193:44605)
0927 07:00:16.529506 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:16.529523 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:16.529685 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:16.530008 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:16.530012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:16.530466 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:16.530749 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:16.530760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.531900 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:16.531906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:16.533541 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:16.533563 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.534312 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:16.534320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:16.534493 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:16.535343 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:16.535361 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:16.536317 (+ 956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:16.539086 (+ 2769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:16.539092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:16.539097 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:16.539581 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:16.540035 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:16.540040 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:16.540047 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:16.540217 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:16.540904 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:16.540909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:16.541146 (+ 237us) client_negotiation.cc:769] Sending connection context
0927 07:00:16.541316 (+ 170us) client_negotiation.cc:240] Negotiation successful
0927 07:00:16.541674 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":633,"threads_started":1}
I20260927 07:00:16.548197 14993 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:33515
I20260927 07:00:16.548231 15128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:33515 every 8 connection(s)
I20260927 07:00:16.549391 14993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:16.551594 15129 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41319
I20260927 07:00:16.551815 15129 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:16.552450 15129 heartbeater.cc:511] Master 127.13.121.254:41319 requested a full tablet report, sending...
I20260927 07:00:16.554421 14922 ts_manager.cc:194] Registered new tserver with Master: d90077940d6847a8958c0c81371c6cf8 (127.13.121.193:33515)
I20260927 07:00:16.555891 14922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:41955
I20260927 07:00:16.556497 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 14993
I20260927 07:00:16.556723 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:16.557336 13799 external_mini_cluster.cc:1428] Setting key 4afe25cc132e3f05d1c01ccefa70564e
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:00:16.595516 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41319
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:34659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:16.907099 15133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:16.907411 15133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:16.907505 15133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:16.915997 15133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:16.916129 15133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:16.916220 15133 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:00:16.916301 15133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:16.926820 15133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:34659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.121.254:41319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:16.928192 15133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:16.929577 15133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:16.943506 15141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:16.943440 15139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:16.943476 15138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:16.944351 15133 server_base.cc:1034] running on GCE node
I20260927 07:00:16.944929 15133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:16.945868 15133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:16.947085 15133 hybrid_clock.cc:648] HybridClock initialized: now 1790492416947036 us; error 52 us; skew 500 ppm
I20260927 07:00:16.947431 15133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:16 dist-test-slave-p6hz krb5kdc[14877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492416, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:16.953855 15133 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:00:16.955394 15133 webserver.cc:460] Webserver started at http://127.13.121.194:41655/ using document root <none> and password file <none>
I20260927 07:00:16.956079 15133 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:16.956178 15133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:16.956492 15133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:16.959447 15133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "26fff2ec2938417791d4148886707a2d"
format_stamp: "Formatted at 2026-09-27 07:00:16 on dist-test-slave-p6hz"
server_key: "90674fff188987f6c4055fe5f068c47d"
server_key_iv: "d0accd1d6807719b43ecb38fa4644d40"
server_key_version: "encryptionkey@0"
I20260927 07:00:16.960203 15133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "26fff2ec2938417791d4148886707a2d"
format_stamp: "Formatted at 2026-09-27 07:00:16 on dist-test-slave-p6hz"
server_key: "90674fff188987f6c4055fe5f068c47d"
server_key_iv: "d0accd1d6807719b43ecb38fa4644d40"
server_key_version: "encryptionkey@0"
I20260927 07:00:16.965394 15133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:00:16.968864 15148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:16.970232 15133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:00:16.970391 15133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "26fff2ec2938417791d4148886707a2d"
format_stamp: "Formatted at 2026-09-27 07:00:16 on dist-test-slave-p6hz"
server_key: "90674fff188987f6c4055fe5f068c47d"
server_key_iv: "d0accd1d6807719b43ecb38fa4644d40"
server_key_version: "encryptionkey@0"
I20260927 07:00:16.970625 15133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:16.982004 15133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:16.986238 15133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:16.986491 15133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:16.989246 15133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:16.989353 15133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:16.989480 15133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:16.989557 15133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:00:16 dist-test-slave-p6hz krb5kdc[14877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492416, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:17.014340 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:16.989775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:40649 (local address 127.13.121.254:41319)
0927 07:00:16.989947 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:16.989954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:16.991577 (+ 1623us) server_negotiation.cc:407] Connection header received
0927 07:00:16.993641 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:16.993647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:16.993759 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:16.993843 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:16.997185 (+ 3342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.998348 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.001239 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.001893 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.002031 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.007570 (+ 5539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:17.007575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:17.007579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:17.007614 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:17.010898 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.012009 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.012016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.012020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.012166 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.012625 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.012630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.012634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.012914 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:17.013052 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.013764 (+ 712us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.013995 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:00:17.015493 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:16.990461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41319 (local address 127.13.121.194:40649)
0927 07:00:16.991408 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:16.991449 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:16.993359 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:16.994123 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:16.994137 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:16.994935 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:16.996981 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:16.997006 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:16.998556 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:16.998562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.001026 (+ 2464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.001043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.002402 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.002411 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.002845 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.003736 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:17.003774 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:17.007320 (+ 3546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:17.011091 (+ 3771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:17.011099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:17.011114 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:17.011821 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:17.012278 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:17.012283 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:17.012287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:17.012502 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:17.013044 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:17.013052 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:17.013416 (+ 364us) client_negotiation.cc:769] Sending connection context
0927 07:00:17.013652 (+ 236us) client_negotiation.cc:240] Negotiation successful
0927 07:00:17.014193 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":377,"threads_started":1}
I20260927 07:00:17.034162 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.021126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:59345 (local address 127.13.121.254:41319)
0927 07:00:17.021281 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.021286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.022039 (+ 753us) server_negotiation.cc:407] Connection header received
0927 07:00:17.022193 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.022197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.022257 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.022337 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:17.023529 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.024367 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.025750 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.026154 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.026262 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.028303 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:17.028310 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:17.028315 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:17.028359 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:17.031424 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.032297 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.032303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.032306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.032432 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.032774 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.032779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.032782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.033132 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:17.033270 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.033632 (+ 362us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.033840 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 07:00:17.034276 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.021276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41319 (local address 127.13.121.194:59345)
0927 07:00:17.021923 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:17.021939 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:17.022078 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:17.022382 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:17.022386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:17.022935 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:17.023325 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.023337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.024531 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.024534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.025622 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.025634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.026325 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.026331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.026477 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.027021 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:17.027038 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:17.028020 (+ 982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:17.031574 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:17.031580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:17.031588 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:17.032177 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:17.032537 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:17.032542 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:17.032546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:17.032668 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:17.033241 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:17.033247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:17.033419 (+ 172us) client_negotiation.cc:769] Sending connection context
0927 07:00:17.033532 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 07:00:17.033864 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":384,"threads_started":1}
I20260927 07:00:17.042524 15133 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:42265
I20260927 07:00:17.042603 15267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:42265 every 8 connection(s)
I20260927 07:00:17.043777 15133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:17.045735 15268 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41319
I20260927 07:00:17.045965 15268 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:17.046651 15268 heartbeater.cc:511] Master 127.13.121.254:41319 requested a full tablet report, sending...
I20260927 07:00:17.048036 14922 ts_manager.cc:194] Registered new tserver with Master: 26fff2ec2938417791d4148886707a2d (127.13.121.194:42265)
I20260927 07:00:17.048846 14922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:40649
I20260927 07:00:17.051288 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 15133
I20260927 07:00:17.051465 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:17.051858 13799 external_mini_cluster.cc:1428] Setting key ba4d65d532a3addcee2f75cfda42ee57
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:00:17.090196 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41319
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:34659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:17.410245 15272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:17.410559 15272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:17.410696 15272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:17.418798 15272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:17.418958 15272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:17.419094 15272 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:00:17.419236 15272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:17.429630 15272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:34659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.121.254:41319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:17.430968 15272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:17.432643 15272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:17.446101 15277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:17.446192 15280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:17.446154 15278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:17.447357 15272 server_base.cc:1034] running on GCE node
I20260927 07:00:17.447993 15272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:17.449088 15272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:17.450345 15272 hybrid_clock.cc:648] HybridClock initialized: now 1790492417450274 us; error 43 us; skew 500 ppm
I20260927 07:00:17.450685 15272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:17 dist-test-slave-p6hz krb5kdc[14877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:17.453970 15272 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:00:17.455613 15272 webserver.cc:460] Webserver started at http://127.13.121.195:44269/ using document root <none> and password file <none>
I20260927 07:00:17.456358 15272 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:17.456462 15272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:17.456777 15272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:17.459792 15272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "789e1faa12984371bb60cba88d0d7290"
format_stamp: "Formatted at 2026-09-27 07:00:17 on dist-test-slave-p6hz"
server_key: "bc23e1f2d47123a5118ab8403c01cf20"
server_key_iv: "440a97211959fb8c74f7f313be9e7f75"
server_key_version: "encryptionkey@0"
I20260927 07:00:17.460547 15272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "789e1faa12984371bb60cba88d0d7290"
format_stamp: "Formatted at 2026-09-27 07:00:17 on dist-test-slave-p6hz"
server_key: "bc23e1f2d47123a5118ab8403c01cf20"
server_key_iv: "440a97211959fb8c74f7f313be9e7f75"
server_key_version: "encryptionkey@0"
I20260927 07:00:17.465893 15272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:00:17.469640 15287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:17.471060 15272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:00:17.471266 15272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "789e1faa12984371bb60cba88d0d7290"
format_stamp: "Formatted at 2026-09-27 07:00:17 on dist-test-slave-p6hz"
server_key: "bc23e1f2d47123a5118ab8403c01cf20"
server_key_iv: "440a97211959fb8c74f7f313be9e7f75"
server_key_version: "encryptionkey@0"
I20260927 07:00:17.471478 15272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:17.490862 15272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:17.495265 15272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:17.495513 15272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:17.497994 15272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:17.498091 15272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:17.498217 15272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:17.498273 15272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:00:17 dist-test-slave-p6hz krb5kdc[14877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492417, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:17.521598 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.498692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:52993 (local address 127.13.121.254:41319)
0927 07:00:17.498862 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.498868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.500297 (+ 1429us) server_negotiation.cc:407] Connection header received
0927 07:00:17.503220 (+ 2923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.503227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.503323 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.503424 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:17.505652 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.506805 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.509143 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.509671 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.509788 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.514496 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:17.514501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:17.514505 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:17.514539 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:17.518012 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.518810 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.518816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.518820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.518992 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.519475 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.519480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.519484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.519852 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:17.520010 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.521001 (+ 991us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.521249 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 07:00:17.523061 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.499265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41319 (local address 127.13.121.195:52993)
0927 07:00:17.500107 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:17.500164 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:17.502705 (+ 2541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:17.503731 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:17.503739 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:17.504535 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:17.505388 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.505421 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.506985 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.506994 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.508930 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.508949 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.509926 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.509934 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.510168 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.510969 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:17.511008 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:17.514265 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:17.518164 (+ 3899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:17.518174 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:17.518192 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:17.518682 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:17.519085 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:17.519088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:17.519091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:17.519337 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:17.519984 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:17.519992 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:17.520511 (+ 519us) client_negotiation.cc:769] Sending connection context
0927 07:00:17.520871 (+ 360us) client_negotiation.cc:240] Negotiation successful
0927 07:00:17.521414 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":322,"threads_started":1}
I20260927 07:00:17.542914 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.528949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:52989 (local address 127.13.121.254:41319)
0927 07:00:17.529146 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.529150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.529683 (+ 533us) server_negotiation.cc:407] Connection header received
0927 07:00:17.529845 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.529850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.529928 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.530010 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:17.530954 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.531819 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.533722 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.534396 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.534527 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.537197 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:17.537202 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:17.537205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:17.537239 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:17.540255 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.541110 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.541114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.541117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.541255 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.541640 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.541643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.541645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.541918 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:17.542042 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.542476 (+ 434us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.542674 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 07:00:17.543290 15369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.529026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41319 (local address 127.13.121.195:52989)
0927 07:00:17.529568 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:17.529591 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:17.529732 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:17.530047 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:17.530051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:17.530522 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:17.530807 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.530817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.531982 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.531990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.533538 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.533553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.534789 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.534797 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.534980 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.535741 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:17.535764 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:17.536935 (+ 1171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:17.540400 (+ 3465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:17.540406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:17.540411 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:17.540969 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:17.541357 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:17.541363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:17.541367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:17.541514 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:17.542020 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:17.542025 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:17.542243 (+ 218us) client_negotiation.cc:769] Sending connection context
0927 07:00:17.542388 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 07:00:17.542751 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":303,"threads_started":1}
I20260927 07:00:17.549093 15272 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:32883
I20260927 07:00:17.549127 15406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:32883 every 8 connection(s)
I20260927 07:00:17.550236 15272 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:17.552369 15407 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41319
I20260927 07:00:17.552613 15407 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:17.553296 15407 heartbeater.cc:511] Master 127.13.121.254:41319 requested a full tablet report, sending...
I20260927 07:00:17.554520 14922 ts_manager.cc:194] Registered new tserver with Master: 789e1faa12984371bb60cba88d0d7290 (127.13.121.195:32883)
I20260927 07:00:17.555388 14922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:52993
I20260927 07:00:17.557225 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 15272
I20260927 07:00:17.557412 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:17.557972 13799 external_mini_cluster.cc:1428] Setting key 9609cbd8fe5b098f3ba0926a162be50a
I20260927 07:00:17.558931 15129 heartbeater.cc:503] Master 127.13.121.254:41319 was elected leader, sending a full tablet report...
I20260927 07:00:17.561128 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:00:17 dist-test-slave-p6hz krb5kdc[14877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492417, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:00:17 dist-test-slave-p6hz krb5kdc[14877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492417, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:17.602705 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.589729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54564 (local address 127.13.121.254:41319)
0927 07:00:17.589933 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.589940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.590174 (+ 234us) server_negotiation.cc:407] Connection header received
0927 07:00:17.590313 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.590319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.590385 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.590468 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:17.591956 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.592811 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.594053 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.594493 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.594593 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.598108 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:17.598114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:17.598119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:17.598163 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:17.600589 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.601188 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.601191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.601194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.601306 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:17.601583 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:17.601586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:17.601588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:17.601866 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:17.602042 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.602227 (+ 185us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.602387 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 07:00:17.606963 14921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:00:17.613726 14921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:17.639276 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.626503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.13.121.195:32883)
0927 07:00:17.628152 (+ 1649us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.628162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.628967 (+ 805us) server_negotiation.cc:407] Connection header received
0927 07:00:17.629694 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.629702 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.629956 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.630130 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:17.630796 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.633891 (+ 3095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.636607 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.638137 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.638310 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.638499 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.638738 (+ 239us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.638907 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1470,"thread_start_us":1310,"threads_started":1}
I20260927 07:00:17.639376 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.625992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:32883 (local address 127.0.0.1:54686)
0927 07:00:17.628667 (+ 2675us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:17.628702 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:17.629402 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:17.630142 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:17.630150 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:17.630177 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:17.630550 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.630562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.634053 (+ 3491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.634059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.636407 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.636424 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.638311 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.638316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.638513 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.638564 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 07:00:17.638695 (+ 131us) client_negotiation.cc:240] Negotiation successful
0927 07:00:17.638951 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2440,"mutex_wait_us":36,"thread_start_us":3435,"threads_started":1}
I20260927 07:00:17.640328 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.626335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.13.121.193:33515)
0927 07:00:17.629354 (+ 3019us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.629362 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.629952 (+ 590us) server_negotiation.cc:407] Connection header received
0927 07:00:17.630047 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.630055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.630294 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.630474 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:17.631392 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.633056 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.636869 (+ 3813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.638278 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.638449 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.638638 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.639900 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.640066 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2873,"thread_start_us":2735,"threads_started":1}
I20260927 07:00:17.639707 15420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.626108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:33515 (local address 127.0.0.1:39048)
0927 07:00:17.629712 (+ 3604us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:17.629729 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:17.629842 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:17.630721 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:17.630726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:17.630754 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:17.631201 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.631212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.633245 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.633251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.636702 (+ 3451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.636712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.638783 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.638789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.639032 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.639069 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 07:00:17.639165 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 07:00:17.639301 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3476,"mutex_wait_us":705,"thread_start_us":2201,"threads_started":1}
I20260927 07:00:17.643537 15337 tablet_service.cc:1467] Processing CreateTablet for tablet 498883d4f1f2456dbe6c95404a7a7d25 (DEFAULT_TABLE table=test-table [id=5f448d57aa854402b59f9eaa33dc42c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:17.644248 15059 tablet_service.cc:1467] Processing CreateTablet for tablet 498883d4f1f2456dbe6c95404a7a7d25 (DEFAULT_TABLE table=test-table [id=5f448d57aa854402b59f9eaa33dc42c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:17.644914 15337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 498883d4f1f2456dbe6c95404a7a7d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:17.645251 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.630505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42265 (local address 127.0.0.1:56888)
0927 07:00:17.636384 (+ 5879us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:17.636405 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:17.636518 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:17.637460 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:17.637464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:17.637491 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:17.637871 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.637882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.639672 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.639678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.641808 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.641825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.644480 (+ 2655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.644488 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.644657 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.644689 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:00:17.644786 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 07:00:17.644923 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5739,"thread_start_us":5564,"threads_started":1}
I20260927 07:00:17.645295 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.630083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.13.121.194:42265)
0927 07:00:17.636607 (+ 6524us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.636617 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.636647 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:00:17.636748 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.636757 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.637038 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.637265 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:17.638072 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.639509 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.641998 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.643978 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.644172 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.644339 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.644858 (+ 519us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.645030 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6361,"thread_start_us":6251,"threads_started":1}
I20260927 07:00:17.645776 15059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 498883d4f1f2456dbe6c95404a7a7d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:17.651678 15198 tablet_service.cc:1467] Processing CreateTablet for tablet 498883d4f1f2456dbe6c95404a7a7d25 (DEFAULT_TABLE table=test-table [id=5f448d57aa854402b59f9eaa33dc42c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:17.653050 15198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 498883d4f1f2456dbe6c95404a7a7d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:17.657311 15426 tablet_bootstrap.cc:492] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: Bootstrap starting.
I20260927 07:00:17.659327 15427 tablet_bootstrap.cc:492] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: Bootstrap starting.
I20260927 07:00:17.661065 15426 tablet_bootstrap.cc:654] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:17.662320 15426 log.cc:824] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:17.662637 15427 tablet_bootstrap.cc:654] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:17.663745 15427 log.cc:824] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:17.665892 15426 tablet_bootstrap.cc:492] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: No bootstrap required, opened a new log
I20260927 07:00:17.666169 15430 tablet_bootstrap.cc:492] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: Bootstrap starting.
I20260927 07:00:17.666265 15426 ts_tablet_manager.cc:1397] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260927 07:00:17.668934 15430 tablet_bootstrap.cc:654] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:17.669006 15427 tablet_bootstrap.cc:492] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: No bootstrap required, opened a new log
I20260927 07:00:17.669299 15427 ts_tablet_manager.cc:1397] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: Time spent bootstrapping tablet: real 0.010s user 0.010s sys 0.000s
I20260927 07:00:17.669917 15430 log.cc:824] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:17.672356 15430 tablet_bootstrap.cc:492] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: No bootstrap required, opened a new log
I20260927 07:00:17.672605 15430 ts_tablet_manager.cc:1397] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260927 07:00:17.677240 15426 raft_consensus.cc:348] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } }
I20260927 07:00:17.677752 15426 raft_consensus.cc:374] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:17.677910 15426 raft_consensus.cc:729] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90077940d6847a8958c0c81371c6cf8, State: Initialized, Role: FOLLOWER
I20260927 07:00:17.678629 15426 consensus_queue.cc:260] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } }
I20260927 07:00:17.679703 15427 raft_consensus.cc:348] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } }
I20260927 07:00:17.680199 15427 raft_consensus.cc:374] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:17.680341 15427 raft_consensus.cc:729] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 789e1faa12984371bb60cba88d0d7290, State: Initialized, Role: FOLLOWER
I20260927 07:00:17.680624 15426 ts_tablet_manager.cc:1428] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
I20260927 07:00:17.681015 15427 consensus_queue.cc:260] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } }
I20260927 07:00:17.682121 15407 heartbeater.cc:503] Master 127.13.121.254:41319 was elected leader, sending a full tablet report...
I20260927 07:00:17.682617 15427 ts_tablet_manager.cc:1428] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260927 07:00:17.683571 15430 raft_consensus.cc:348] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } }
I20260927 07:00:17.684024 15430 raft_consensus.cc:374] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:17.684116 15430 raft_consensus.cc:729] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26fff2ec2938417791d4148886707a2d, State: Initialized, Role: FOLLOWER
I20260927 07:00:17.684698 15430 consensus_queue.cc:260] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } }
I20260927 07:00:17.685724 15268 heartbeater.cc:503] Master 127.13.121.254:41319 was elected leader, sending a full tablet report...
I20260927 07:00:17.687572 15430 ts_tablet_manager.cc:1428] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: Time spent starting tablet: real 0.015s user 0.011s sys 0.000s
I20260927 07:00:17.698040 15434 raft_consensus.cc:482] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:17.698333 15434 raft_consensus.cc:504] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } }
I20260927 07:00:17.700008 15434 leader_election.cc:288] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 789e1faa12984371bb60cba88d0d7290 (127.13.121.195:32883), d90077940d6847a8958c0c81371c6cf8 (127.13.121.193:33515)
I20260927 07:00:17.706480 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.700422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:58923 (local address 127.13.121.193:33515)
0927 07:00:17.700804 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.700810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.701267 (+ 457us) server_negotiation.cc:407] Connection header received
0927 07:00:17.701457 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.701461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.701540 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.701623 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:17.702143 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.702826 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.704815 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.705629 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.705775 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.705849 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.706195 (+ 346us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.706302 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260927 07:00:17.706676 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.700539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:50379 (local address 127.13.121.195:32883)
0927 07:00:17.700757 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.700763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.701539 (+ 776us) server_negotiation.cc:407] Connection header received
0927 07:00:17.701704 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.701708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.701765 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.701838 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:17.702384 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.703078 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.704962 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.705743 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.705884 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.705952 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.706318 (+ 366us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.706434 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260927 07:00:17.706790 15435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.700462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:32883 (local address 127.13.121.194:50379)
0927 07:00:17.701437 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:17.701451 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:17.701560 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:17.701865 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:17.701873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:17.701896 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:17.702236 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.702245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.703226 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.703231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.704820 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.704834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.705942 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.705948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.706171 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.706203 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:00:17.706280 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 07:00:17.706409 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":3,"thread_start_us":433,"threads_started":1}
I20260927 07:00:17.707129 15436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.700310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:33515 (local address 127.13.121.194:58923)
0927 07:00:17.701156 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:17.701173 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:17.701330 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:17.701650 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:17.701655 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:17.701679 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:17.701995 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.702008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.702946 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.702950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.704658 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:17.704672 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.705822 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.705827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:17.706019 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.706042 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 07:00:17.706676 (+ 634us) client_negotiation.cc:240] Negotiation successful
0927 07:00:17.706809 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":55,"thread_start_us":829,"threads_started":1}
I20260927 07:00:17.707803 15357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "498883d4f1f2456dbe6c95404a7a7d25" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789e1faa12984371bb60cba88d0d7290" is_pre_election: true
I20260927 07:00:17.708096 15079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "498883d4f1f2456dbe6c95404a7a7d25" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90077940d6847a8958c0c81371c6cf8" is_pre_election: true
I20260927 07:00:17.708390 15357 raft_consensus.cc:2454] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 0.
I20260927 07:00:17.708586 15079 raft_consensus.cc:2454] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 0.
I20260927 07:00:17.709136 15150 leader_election.cc:302] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26fff2ec2938417791d4148886707a2d, 789e1faa12984371bb60cba88d0d7290; no voters:
I20260927 07:00:17.709580 15434 raft_consensus.cc:2781] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:17.709718 15434 raft_consensus.cc:482] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:17.709827 15434 raft_consensus.cc:3037] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:17.712157 15434 raft_consensus.cc:504] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } }
I20260927 07:00:17.712723 15434 leader_election.cc:288] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 election: Requested vote from peers 789e1faa12984371bb60cba88d0d7290 (127.13.121.195:32883), d90077940d6847a8958c0c81371c6cf8 (127.13.121.193:33515)
I20260927 07:00:17.713263 15357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "498883d4f1f2456dbe6c95404a7a7d25" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789e1faa12984371bb60cba88d0d7290"
I20260927 07:00:17.713305 15079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "498883d4f1f2456dbe6c95404a7a7d25" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90077940d6847a8958c0c81371c6cf8"
I20260927 07:00:17.713491 15357 raft_consensus.cc:3037] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:17.713491 15079 raft_consensus.cc:3037] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:17.716750 15357 raft_consensus.cc:2454] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 1.
I20260927 07:00:17.716735 15079 raft_consensus.cc:2454] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 1.
I20260927 07:00:17.717348 15149 leader_election.cc:302] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26fff2ec2938417791d4148886707a2d, d90077940d6847a8958c0c81371c6cf8; no voters:
I20260927 07:00:17.717687 15434 raft_consensus.cc:2781] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:17.718154 15434 raft_consensus.cc:686] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 1 LEADER]: Becoming Leader. State: Replica: 26fff2ec2938417791d4148886707a2d, State: Running, Role: LEADER
I20260927 07:00:17.718684 15434 consensus_queue.cc:237] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } }
I20260927 07:00:17.723956 14921 catalog_manager.cc:5495] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 26fff2ec2938417791d4148886707a2d (127.13.121.194). New cstate: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } health_report { overall_health: HEALTHY } } }
I20260927 07:00:17.767095 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:17.761704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.13.121.194:42265)
0927 07:00:17.761908 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:17.761915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:17.762002 (+ 87us) server_negotiation.cc:407] Connection header received
0927 07:00:17.762330 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:17.762334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:17.762411 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:17.762530 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:17.762887 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.763700 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.764996 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:17.765389 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:17.765496 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:17.765977 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:17.766507 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:17.766636 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:17.766764 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 07:00:17.766846 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:00:17.775509 15357 raft_consensus.cc:1261] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Refusing update from remote peer 26fff2ec2938417791d4148886707a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:17.775520 15079 raft_consensus.cc:1261] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Refusing update from remote peer 26fff2ec2938417791d4148886707a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:17.776660 15434 consensus_queue.cc:1035] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:17.777006 15437 consensus_queue.cc:1035] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:17.786253 15440 mvcc.cc:205] Tried to move back new op lower bound from 7333856943198027776 to 7333856942979960832. Current Snapshot: MvccSnapshot[applied={T|T < 7333856943198027776}]
I20260927 07:00:17.788913 15441 mvcc.cc:205] Tried to move back new op lower bound from 7333856943198027776 to 7333856942979960832. Current Snapshot: MvccSnapshot[applied={T|T < 7333856943198027776}]
I20260927 07:00:17.792699 15442 mvcc.cc:205] Tried to move back new op lower bound from 7333856943198027776 to 7333856942979960832. Current Snapshot: MvccSnapshot[applied={T|T < 7333856943198027776}]
I20260927 07:00:17.808801 14921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:17.809029 14921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:17.812417 14921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381: Sending DeleteTablet for 3 replicas of tablet 498883d4f1f2456dbe6c95404a7a7d25
I20260927 07:00:17.813674 15337 tablet_service.cc:1514] Processing DeleteTablet for tablet 498883d4f1f2456dbe6c95404a7a7d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:17 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:54686
I20260927 07:00:17.813654 15059 tablet_service.cc:1514] Processing DeleteTablet for tablet 498883d4f1f2456dbe6c95404a7a7d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:17 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:39048
I20260927 07:00:17.813653 15198 tablet_service.cc:1514] Processing DeleteTablet for tablet 498883d4f1f2456dbe6c95404a7a7d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:17 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:56888
I20260927 07:00:17.815507 14922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:00:17.816290 14922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:17.819090 15452 tablet_replica.cc:330] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: stopping tablet replica
I20260927 07:00:17.819703 15452 raft_consensus.cc:2229] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:17.820309 15452 raft_consensus.cc:2258] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:17.821152 15451 tablet_replica.cc:330] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: stopping tablet replica
I20260927 07:00:17.821615 15451 raft_consensus.cc:2229] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:17.823721 15452 ts_tablet_manager.cc:1905] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:17.824142 15453 tablet_replica.cc:330] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: stopping tablet replica
I20260927 07:00:17.825659 15059 tablet_service.cc:1467] Processing CreateTablet for tablet d755c680e8f74890bfea6f5af1bb03ce (DEFAULT_TABLE table=test-table [id=937c0769f34d48079716255f3ce51048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:17.825758 15337 tablet_service.cc:1467] Processing CreateTablet for tablet d755c680e8f74890bfea6f5af1bb03ce (DEFAULT_TABLE table=test-table [id=937c0769f34d48079716255f3ce51048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:17.826145 15059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d755c680e8f74890bfea6f5af1bb03ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:17.826159 15337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d755c680e8f74890bfea6f5af1bb03ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:17.826231 15451 raft_consensus.cc:2258] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:17.832230 15426 tablet_bootstrap.cc:492] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: Bootstrap starting.
I20260927 07:00:17.832798 15452 ts_tablet_manager.cc:1918] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:00:17.832952 15452 log.cc:1196] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/498883d4f1f2456dbe6c95404a7a7d25
I20260927 07:00:17.833454 15452 ts_tablet_manager.cc:1939] T 498883d4f1f2456dbe6c95404a7a7d25 P 789e1faa12984371bb60cba88d0d7290: Deleting consensus metadata
I20260927 07:00:17.833504 15426 tablet_bootstrap.cc:654] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:17.835426 14907 catalog_manager.cc:4841] TS 789e1faa12984371bb60cba88d0d7290 (127.13.121.195:32883): tablet 498883d4f1f2456dbe6c95404a7a7d25 (table test-table [id=5f448d57aa854402b59f9eaa33dc42c8]) successfully deleted
I20260927 07:00:17.836193 15427 tablet_bootstrap.cc:492] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: Bootstrap starting.
I20260927 07:00:17.824774 15453 raft_consensus.cc:2229] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:00:17.825285 15198 tablet_service.cc:1467] Processing CreateTablet for tablet d755c680e8f74890bfea6f5af1bb03ce (DEFAULT_TABLE table=test-table [id=937c0769f34d48079716255f3ce51048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:17.836972 15453 raft_consensus.cc:2258] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:17.837421 15198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d755c680e8f74890bfea6f5af1bb03ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:17.837759 15427 tablet_bootstrap.cc:654] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:17.838295 15426 tablet_bootstrap.cc:492] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: No bootstrap required, opened a new log
I20260927 07:00:17.838433 15426 ts_tablet_manager.cc:1397] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260927 07:00:17.839231 15426 raft_consensus.cc:348] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:17.839453 15426 raft_consensus.cc:374] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:17.839543 15426 raft_consensus.cc:729] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90077940d6847a8958c0c81371c6cf8, State: Initialized, Role: FOLLOWER
I20260927 07:00:17.839962 15426 consensus_queue.cc:260] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:17.843351 15426 ts_tablet_manager.cc:1428] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260927 07:00:17.845765 15451 ts_tablet_manager.cc:1905] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:17.846980 15430 tablet_bootstrap.cc:492] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: Bootstrap starting.
I20260927 07:00:17.848325 15427 tablet_bootstrap.cc:492] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: No bootstrap required, opened a new log
I20260927 07:00:17.848526 15427 ts_tablet_manager.cc:1397] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260927 07:00:17.848526 15453 ts_tablet_manager.cc:1905] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:17.848856 15430 tablet_bootstrap.cc:654] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:17.849330 15427 raft_consensus.cc:348] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:17.849542 15427 raft_consensus.cc:374] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:17.849633 15427 raft_consensus.cc:729] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 789e1faa12984371bb60cba88d0d7290, State: Initialized, Role: FOLLOWER
I20260927 07:00:17.849938 15427 consensus_queue.cc:260] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:17.853885 15427 ts_tablet_manager.cc:1428] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260927 07:00:17.856061 15453 ts_tablet_manager.cc:1918] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:00:17.856256 15453 log.cc:1196] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/498883d4f1f2456dbe6c95404a7a7d25
I20260927 07:00:17.856806 15453 ts_tablet_manager.cc:1939] T 498883d4f1f2456dbe6c95404a7a7d25 P 26fff2ec2938417791d4148886707a2d: Deleting consensus metadata
I20260927 07:00:17.858346 14907 catalog_manager.cc:4841] TS 26fff2ec2938417791d4148886707a2d (127.13.121.194:42265): tablet 498883d4f1f2456dbe6c95404a7a7d25 (table test-table [id=5f448d57aa854402b59f9eaa33dc42c8]) successfully deleted
I20260927 07:00:17.861398 15451 ts_tablet_manager.cc:1918] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:00:17.861552 15451 log.cc:1196] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/498883d4f1f2456dbe6c95404a7a7d25
I20260927 07:00:17.861981 15451 ts_tablet_manager.cc:1939] T 498883d4f1f2456dbe6c95404a7a7d25 P d90077940d6847a8958c0c81371c6cf8: Deleting consensus metadata
I20260927 07:00:17.863481 14906 catalog_manager.cc:4841] TS d90077940d6847a8958c0c81371c6cf8 (127.13.121.193:33515): tablet 498883d4f1f2456dbe6c95404a7a7d25 (table test-table [id=5f448d57aa854402b59f9eaa33dc42c8]) successfully deleted
I20260927 07:00:17.864773 15430 tablet_bootstrap.cc:492] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: No bootstrap required, opened a new log
I20260927 07:00:17.864938 15430 ts_tablet_manager.cc:1397] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.007s
I20260927 07:00:17.865777 15430 raft_consensus.cc:348] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:17.866019 15430 raft_consensus.cc:374] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:17.866151 15430 raft_consensus.cc:729] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26fff2ec2938417791d4148886707a2d, State: Initialized, Role: FOLLOWER
I20260927 07:00:17.866420 15430 consensus_queue.cc:260] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:17.867107 15430 ts_tablet_manager.cc:1428] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260927 07:00:17.947985 15447 raft_consensus.cc:482] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:17.948215 15447 raft_consensus.cc:504] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:17.948953 15447 leader_election.cc:288] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d90077940d6847a8958c0c81371c6cf8 (127.13.121.193:33515), 789e1faa12984371bb60cba88d0d7290 (127.13.121.195:32883)
I20260927 07:00:17.949532 15079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d755c680e8f74890bfea6f5af1bb03ce" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90077940d6847a8958c0c81371c6cf8" is_pre_election: true
I20260927 07:00:17.949532 15357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d755c680e8f74890bfea6f5af1bb03ce" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789e1faa12984371bb60cba88d0d7290" is_pre_election: true
I20260927 07:00:17.949777 15357 raft_consensus.cc:2454] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 0.
I20260927 07:00:17.949793 15079 raft_consensus.cc:2454] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 0.
I20260927 07:00:17.950201 15150 leader_election.cc:302] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26fff2ec2938417791d4148886707a2d, 789e1faa12984371bb60cba88d0d7290; no voters:
I20260927 07:00:17.950486 15447 raft_consensus.cc:2781] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:17.950620 15447 raft_consensus.cc:482] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:17.950717 15447 raft_consensus.cc:3037] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:17.953890 15447 raft_consensus.cc:504] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:17.954562 15447 leader_election.cc:288] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 election: Requested vote from peers d90077940d6847a8958c0c81371c6cf8 (127.13.121.193:33515), 789e1faa12984371bb60cba88d0d7290 (127.13.121.195:32883)
I20260927 07:00:17.955055 15079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d755c680e8f74890bfea6f5af1bb03ce" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90077940d6847a8958c0c81371c6cf8"
I20260927 07:00:17.955096 15357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d755c680e8f74890bfea6f5af1bb03ce" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789e1faa12984371bb60cba88d0d7290"
I20260927 07:00:17.955271 15079 raft_consensus.cc:3037] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:17.955318 15357 raft_consensus.cc:3037] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:17.957427 15079 raft_consensus.cc:2454] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 1.
I20260927 07:00:17.957856 15149 leader_election.cc:302] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26fff2ec2938417791d4148886707a2d, d90077940d6847a8958c0c81371c6cf8; no voters:
I20260927 07:00:17.958154 15447 raft_consensus.cc:2781] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:17.958215 15357 raft_consensus.cc:2454] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 1.
I20260927 07:00:17.958365 15447 raft_consensus.cc:686] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 1 LEADER]: Becoming Leader. State: Replica: 26fff2ec2938417791d4148886707a2d, State: Running, Role: LEADER
I20260927 07:00:17.958688 15447 consensus_queue.cc:237] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:17.961553 14922 catalog_manager.cc:5495] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 26fff2ec2938417791d4148886707a2d (127.13.121.194). New cstate: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } health_report { overall_health: UNKNOWN } } }
Sep 27 07:00:18 dist-test-slave-p6hz krb5kdc[14877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492415, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:18.029367 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.013613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41319 (local address 127.0.0.1:54576)
0927 07:00:18.014267 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:18.014287 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:18.014426 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:18.014826 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:18.014830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:18.015557 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:18.015856 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:18.015865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.017255 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.017260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:18.018340 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:18.018352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.019358 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.019362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:18.019494 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.020141 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:18.020157 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:18.022483 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:18.026787 (+ 4304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:18.026799 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:18.026805 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:18.027295 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:18.027734 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:18.027737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:18.027739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:18.027869 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:18.028478 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:18.028485 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:18.028672 (+ 187us) client_negotiation.cc:769] Sending connection context
0927 07:00:18.028818 (+ 146us) client_negotiation.cc:240] Negotiation successful
0927 07:00:18.029047 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":342,"threads_started":1}
I20260927 07:00:18.029709 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.013601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54576 (local address 127.13.121.254:41319)
0927 07:00:18.013814 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:18.013821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:18.014385 (+ 564us) server_negotiation.cc:407] Connection header received
0927 07:00:18.014533 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:18.014539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:18.014651 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:18.014771 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:18.016020 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.017115 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.018486 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.019187 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.019310 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.022701 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:18.022707 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:18.022711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:18.022750 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:18.026615 (+ 3865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.027413 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.027419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.027422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.027593 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.027967 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.027971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.027974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.028323 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:18.028467 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:18.028939 (+ 472us) server_negotiation.cc:299] Negotiation successful
0927 07:00:18.029293 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":65}
I20260927 07:00:18.035006 14922 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:54576:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 07:00:18.036186 14922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:18.043795 15059 tablet_service.cc:1467] Processing CreateTablet for tablet 7b45477d7b5f42ad99092a9a9f89d340 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b5e1559af9ed4c30ac294615913d59df]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:00:18.044401 15059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b45477d7b5f42ad99092a9a9f89d340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:18.044461 15198 tablet_service.cc:1467] Processing CreateTablet for tablet 7b45477d7b5f42ad99092a9a9f89d340 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b5e1559af9ed4c30ac294615913d59df]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:00:18.045042 15198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b45477d7b5f42ad99092a9a9f89d340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:18.048578 15337 tablet_service.cc:1467] Processing CreateTablet for tablet 7b45477d7b5f42ad99092a9a9f89d340 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b5e1559af9ed4c30ac294615913d59df]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:00:18.049114 15337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b45477d7b5f42ad99092a9a9f89d340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:18.052120 15426 tablet_bootstrap.cc:492] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: Bootstrap starting.
W20260927 07:00:18.052497 15130 tablet.cc:2369] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:00:18.053051 15269 tablet.cc:2369] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:00:18.054037 15426 tablet_bootstrap.cc:654] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: Neither blocks nor log segments found. Creating new log.
W20260927 07:00:18.055035 15408 tablet.cc:2369] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:00:18.062428 15426 tablet_bootstrap.cc:492] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: No bootstrap required, opened a new log
I20260927 07:00:18.062716 15426 ts_tablet_manager.cc:1397] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260927 07:00:18.063540 15426 raft_consensus.cc:348] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:18.063884 15426 raft_consensus.cc:374] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:18.064003 15426 raft_consensus.cc:729] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d90077940d6847a8958c0c81371c6cf8, State: Initialized, Role: FOLLOWER
I20260927 07:00:18.064267 15427 tablet_bootstrap.cc:492] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: Bootstrap starting.
I20260927 07:00:18.064334 15426 consensus_queue.cc:260] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:18.064757 15432 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.065048 15432 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:18.065938 15427 tablet_bootstrap.cc:654] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:18.068776 15426 ts_tablet_manager.cc:1428] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260927 07:00:18.071034 15430 tablet_bootstrap.cc:492] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: Bootstrap starting.
I20260927 07:00:18.072904 15430 tablet_bootstrap.cc:654] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:18.075805 15430 tablet_bootstrap.cc:492] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: No bootstrap required, opened a new log
I20260927 07:00:18.075994 15427 tablet_bootstrap.cc:492] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: No bootstrap required, opened a new log
I20260927 07:00:18.076162 15430 ts_tablet_manager.cc:1397] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 07:00:18.076177 15427 ts_tablet_manager.cc:1397] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260927 07:00:18.076901 15427 raft_consensus.cc:348] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:18.076901 15430 raft_consensus.cc:348] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:18.077149 15427 raft_consensus.cc:374] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:18.077167 15430 raft_consensus.cc:374] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:18.077240 15430 raft_consensus.cc:729] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26fff2ec2938417791d4148886707a2d, State: Initialized, Role: FOLLOWER
I20260927 07:00:18.077279 15427 raft_consensus.cc:729] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 789e1faa12984371bb60cba88d0d7290, State: Initialized, Role: FOLLOWER
I20260927 07:00:18.077536 15427 consensus_queue.cc:260] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:18.077549 15430 consensus_queue.cc:260] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:18.077973 15447 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.077989 15433 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.078241 15433 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:18.078269 15447 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:18.078658 15427 ts_tablet_manager.cc:1428] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 07:00:18.080154 15430 ts_tablet_manager.cc:1428] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260927 07:00:18.100077 15447 raft_consensus.cc:482] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:18.100319 15447 raft_consensus.cc:504] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:18.101150 15447 leader_election.cc:288] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d90077940d6847a8958c0c81371c6cf8 (127.13.121.193:33515), 789e1faa12984371bb60cba88d0d7290 (127.13.121.195:32883)
I20260927 07:00:18.101763 15079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b45477d7b5f42ad99092a9a9f89d340" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90077940d6847a8958c0c81371c6cf8" is_pre_election: true
I20260927 07:00:18.102046 15079 raft_consensus.cc:2454] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 0.
I20260927 07:00:18.102201 15357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b45477d7b5f42ad99092a9a9f89d340" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789e1faa12984371bb60cba88d0d7290" is_pre_election: true
I20260927 07:00:18.102468 15357 raft_consensus.cc:2454] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 0.
I20260927 07:00:18.102600 15149 leader_election.cc:302] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26fff2ec2938417791d4148886707a2d, d90077940d6847a8958c0c81371c6cf8; no voters:
I20260927 07:00:18.103328 15447 raft_consensus.cc:2781] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:18.103440 15447 raft_consensus.cc:482] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:18.103556 15447 raft_consensus.cc:3037] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:18.106357 15447 raft_consensus.cc:504] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:18.107254 15447 leader_election.cc:288] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 election: Requested vote from peers d90077940d6847a8958c0c81371c6cf8 (127.13.121.193:33515), 789e1faa12984371bb60cba88d0d7290 (127.13.121.195:32883)
I20260927 07:00:18.107406 15079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b45477d7b5f42ad99092a9a9f89d340" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d90077940d6847a8958c0c81371c6cf8"
I20260927 07:00:18.107666 15079 raft_consensus.cc:3037] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:18.107769 15357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b45477d7b5f42ad99092a9a9f89d340" candidate_uuid: "26fff2ec2938417791d4148886707a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789e1faa12984371bb60cba88d0d7290"
I20260927 07:00:18.107973 15357 raft_consensus.cc:3037] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:18.110605 15079 raft_consensus.cc:2454] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 1.
I20260927 07:00:18.110750 15357 raft_consensus.cc:2454] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26fff2ec2938417791d4148886707a2d in term 1.
I20260927 07:00:18.111387 15150 leader_election.cc:302] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26fff2ec2938417791d4148886707a2d, 789e1faa12984371bb60cba88d0d7290; no voters:
I20260927 07:00:18.111686 15447 raft_consensus.cc:2781] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:18.111850 15447 raft_consensus.cc:686] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [term 1 LEADER]: Becoming Leader. State: Replica: 26fff2ec2938417791d4148886707a2d, State: Running, Role: LEADER
I20260927 07:00:18.112079 15447 consensus_queue.cc:237] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } }
I20260927 07:00:18.113490 15437 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: TxnStatusTablet state changed. Reason: New leader 26fff2ec2938417791d4148886707a2d. Latest consensus state: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.113852 15437 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:00:18.114722 15468 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 07:00:18.114952 15468 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 07:00:18.115607 14922 catalog_manager.cc:5495] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 26fff2ec2938417791d4148886707a2d (127.13.121.194). New cstate: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } health_report { overall_health: UNKNOWN } } }
I20260927 07:00:18.135823 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.130266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56908 (local address 127.13.121.194:42265)
0927 07:00:18.130447 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:18.130453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:18.130527 (+ 74us) server_negotiation.cc:407] Connection header received
0927 07:00:18.130794 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:18.130798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:18.130863 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:18.130944 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:18.131413 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.132197 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.134019 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.134469 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.134577 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.135099 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:18.135258 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:18.135333 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:18.135511 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 07:00:18.135582 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:00:18.135963 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.130164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42265 (local address 127.0.0.1:56908)
0927 07:00:18.130425 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:18.130449 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:18.130612 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:18.130996 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:18.131001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:18.131014 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:00:18.131254 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:18.131266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.132334 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.132340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:18.133875 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:18.133892 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.134636 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.134642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:18.134848 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.134932 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:00:18.135365 (+ 433us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:00:18.135396 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 07:00:18.135492 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 07:00:18.135655 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260927 07:00:18.140019 15079 raft_consensus.cc:1261] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Refusing update from remote peer 26fff2ec2938417791d4148886707a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:18.140163 15357 raft_consensus.cc:1261] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Refusing update from remote peer 26fff2ec2938417791d4148886707a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:18.140733 15437 consensus_queue.cc:1035] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:18.141018 15447 consensus_queue.cc:1035] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:18.143966 15432 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: TxnStatusTablet state changed. Reason: New leader 26fff2ec2938417791d4148886707a2d. Latest consensus state: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.144268 15432 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:18.145884 15447 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.146200 15447 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:00:18.147584 15432 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.147806 15432 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P d90077940d6847a8958c0c81371c6cf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:18.148845 15434 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.149103 15434 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P 26fff2ec2938417791d4148886707a2d: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:00:18.149180 15433 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: TxnStatusTablet state changed. Reason: New leader 26fff2ec2938417791d4148886707a2d. Latest consensus state: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.149451 15433 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:18.151949 15433 tablet_replica.cc:439] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "26fff2ec2938417791d4148886707a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 } } peers { permanent_uuid: "26fff2ec2938417791d4148886707a2d" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42265 } } peers { permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 } } }
I20260927 07:00:18.152216 15433 tablet_replica.cc:442] T 7b45477d7b5f42ad99092a9a9f89d340 P 789e1faa12984371bb60cba88d0d7290: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:18.172696 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.166965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:39581 (local address 127.13.121.194:42265)
0927 07:00:18.167253 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:18.167257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:18.167794 (+ 537us) server_negotiation.cc:407] Connection header received
0927 07:00:18.167974 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:18.167979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:18.168038 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:18.168115 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:18.168591 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.169320 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.170760 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.171273 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.171381 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.171825 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:18.171968 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:18.172045 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:18.172174 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 07:00:18.172324 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":100}
I20260927 07:00:18.172755 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.166923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42265 (local address 127.13.121.194:39581)
0927 07:00:18.167683 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:18.167703 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:18.167851 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:18.168135 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:18.168139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:18.168148 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:00:18.168451 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:18.168463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.169437 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.169442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:18.170621 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:18.170633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.171451 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.171458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:18.171590 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.171657 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:00:18.172062 (+ 405us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:00:18.172071 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 07:00:18.172156 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 07:00:18.172363 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":34,"thread_start_us":447,"threads_started":1}
I20260927 07:00:18.183590 15357 raft_consensus.cc:1261] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Refusing update from remote peer 26fff2ec2938417791d4148886707a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:18.183645 15079 raft_consensus.cc:1261] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Refusing update from remote peer 26fff2ec2938417791d4148886707a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:18.184196 15437 consensus_queue.cc:1035] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d90077940d6847a8958c0c81371c6cf8" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:18.184516 15448 consensus_queue.cc:1035] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "789e1faa12984371bb60cba88d0d7290" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 32883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:18.244400 14921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:18.244673 14921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:18.247664 14921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b85b1b44f964913a616874a0cc49381: Sending DeleteTablet for 3 replicas of tablet d755c680e8f74890bfea6f5af1bb03ce
I20260927 07:00:18.248461 15059 tablet_service.cc:1514] Processing DeleteTablet for tablet d755c680e8f74890bfea6f5af1bb03ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:18 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:39048
I20260927 07:00:18.248644 15337 tablet_service.cc:1514] Processing DeleteTablet for tablet d755c680e8f74890bfea6f5af1bb03ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:18 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:54686
I20260927 07:00:18.248929 15452 tablet_replica.cc:330] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: stopping tablet replica
I20260927 07:00:18.249128 15452 raft_consensus.cc:2229] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:18.249348 15452 raft_consensus.cc:2258] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:18.249730 15451 tablet_replica.cc:330] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: stopping tablet replica
I20260927 07:00:18.249933 15451 raft_consensus.cc:2229] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:18.250087 15451 raft_consensus.cc:2258] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:18.250748 15452 ts_tablet_manager.cc:1905] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:18.251307 15451 ts_tablet_manager.cc:1905] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:18.256628 15198 tablet_service.cc:1514] Processing DeleteTablet for tablet d755c680e8f74890bfea6f5af1bb03ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:18 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:56888
I20260927 07:00:18.256932 15453 tablet_replica.cc:330] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: stopping tablet replica
I20260927 07:00:18.257087 15453 raft_consensus.cc:2229] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:00:18.257390 15453 raft_consensus.cc:2258] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:18.258536 15451 ts_tablet_manager.cc:1918] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:00:18.258582 15453 ts_tablet_manager.cc:1905] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:18.258674 15451 log.cc:1196] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/d755c680e8f74890bfea6f5af1bb03ce
I20260927 07:00:18.259084 15451 ts_tablet_manager.cc:1939] T d755c680e8f74890bfea6f5af1bb03ce P d90077940d6847a8958c0c81371c6cf8: Deleting consensus metadata
I20260927 07:00:18.265455 15453 ts_tablet_manager.cc:1918] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:00:18.265627 15453 log.cc:1196] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/d755c680e8f74890bfea6f5af1bb03ce
I20260927 07:00:18.266016 15453 ts_tablet_manager.cc:1939] T d755c680e8f74890bfea6f5af1bb03ce P 26fff2ec2938417791d4148886707a2d: Deleting consensus metadata
I20260927 07:00:18.266167 14906 catalog_manager.cc:4841] TS d90077940d6847a8958c0c81371c6cf8 (127.13.121.193:33515): tablet d755c680e8f74890bfea6f5af1bb03ce (table test-table [id=937c0769f34d48079716255f3ce51048]) successfully deleted
I20260927 07:00:18.267446 15452 ts_tablet_manager.cc:1918] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:00:18.267634 15452 log.cc:1196] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/d755c680e8f74890bfea6f5af1bb03ce
I20260927 07:00:18.268018 15452 ts_tablet_manager.cc:1939] T d755c680e8f74890bfea6f5af1bb03ce P 789e1faa12984371bb60cba88d0d7290: Deleting consensus metadata
I20260927 07:00:18.269110 14907 catalog_manager.cc:4841] TS 789e1faa12984371bb60cba88d0d7290 (127.13.121.195:32883): tablet d755c680e8f74890bfea6f5af1bb03ce (table test-table [id=937c0769f34d48079716255f3ce51048]) successfully deleted
I20260927 07:00:18.269809 14907 catalog_manager.cc:4841] TS 26fff2ec2938417791d4148886707a2d (127.13.121.194:42265): tablet d755c680e8f74890bfea6f5af1bb03ce (table test-table [id=937c0769f34d48079716255f3ce51048]) successfully deleted
Sep 27 07:00:18 dist-test-slave-p6hz krb5kdc[14877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492417, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:18.283088 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.251032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39062 (local address 127.13.121.193:33515)
0927 07:00:18.266501 (+ 15469us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:18.266508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:18.266996 (+ 488us) server_negotiation.cc:407] Connection header received
0927 07:00:18.267307 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:18.267312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:18.267420 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:18.267497 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:18.268977 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.269847 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.271099 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.271874 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.272013 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.276262 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:18.276282 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:18.276301 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:18.276348 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:18.279990 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.280753 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.280764 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.280768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.281049 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.281458 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.281464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.281466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.281927 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:18.282245 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:18.282424 (+ 179us) server_negotiation.cc:299] Negotiation successful
0927 07:00:18.282683 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14893,"thread_start_us":14813,"threads_started":1}
W20260927 07:00:18.284013 15124 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39062
I20260927 07:00:18.300207 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.286710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.13.121.254:41319)
0927 07:00:18.286922 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:18.286928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:18.287251 (+ 323us) server_negotiation.cc:407] Connection header received
0927 07:00:18.287478 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:18.287484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:18.287560 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:18.287656 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:18.288935 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.290053 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.291266 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.291923 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.292059 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.294256 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:18.294262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:18.294267 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:18.294305 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:18.297531 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.298155 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.298166 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.298170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.298304 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.298609 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.298614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.298618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.298969 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:18.299306 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:18.299633 (+ 327us) server_negotiation.cc:299] Negotiation successful
0927 07:00:18.299847 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260927 07:00:18.300791 14921 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54580
I20260927 07:00:18.302497 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 14993
I20260927 07:00:18.316430 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.303616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.13.121.193:33515)
0927 07:00:18.303855 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:18.303861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:18.304269 (+ 408us) server_negotiation.cc:407] Connection header received
0927 07:00:18.304388 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:18.304393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:18.304468 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:18.304563 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:18.305654 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.306553 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.307794 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.308433 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.308554 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.310518 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:18.310525 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:18.310530 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:18.310581 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:18.313754 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.314476 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.314482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.314486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.314636 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.315001 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.315006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.315009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.315388 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:18.315644 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:18.315874 (+ 230us) server_negotiation.cc:299] Negotiation successful
0927 07:00:18.316095 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260927 07:00:18.317040 15124 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39078
W20260927 07:00:18.317600 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 14993: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:00:18.317806 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 14993
I20260927 07:00:18.337442 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15133
Sep 27 07:00:18 dist-test-slave-p6hz krb5kdc[14877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492417, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:00:18.353780 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.338139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.13.121.194:42265)
0927 07:00:18.338325 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:18.338331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:18.338437 (+ 106us) server_negotiation.cc:407] Connection header received
0927 07:00:18.338625 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:18.338629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:18.338721 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:18.338822 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:18.340014 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.341149 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.342382 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.343120 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.343292 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.347097 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:18.347115 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:18.347180 (+ 65us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:18.347234 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:18.350974 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.351655 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.351665 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.351668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.351823 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.352156 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.352170 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.352173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.352646 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:18.352960 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:18.353137 (+ 177us) server_negotiation.cc:299] Negotiation successful
0927 07:00:18.353375 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260927 07:00:18.354504 15263 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56916
W20260927 07:00:18.355190 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15133: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:00:18.355365 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 15133
I20260927 07:00:18.373020 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15272
Sep 27 07:00:18 dist-test-slave-p6hz krb5kdc[14877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492417, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:00:18.388275 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:18.373756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.13.121.195:32883)
0927 07:00:18.374259 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:18.374264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:18.374280 (+ 16us) server_negotiation.cc:407] Connection header received
0927 07:00:18.374330 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:18.374333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:18.374406 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:18.374498 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:18.375918 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.376738 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.378083 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:18.378523 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:18.378652 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:18.382851 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:18.382868 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:18.382885 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:18.382924 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:18.385506 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.386240 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.386249 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.386251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.386401 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:18.386739 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:18.386742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:18.386744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:18.387087 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:18.387427 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:18.387740 (+ 313us) server_negotiation.cc:299] Negotiation successful
0927 07:00:18.387948 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":259,"threads_started":1}
W20260927 07:00:18.388922 15402 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54694
W20260927 07:00:18.389545 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15272: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:00:18.389683 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 15272
I20260927 07:00:18.404371 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 14890
I20260927 07:00:18.405455 14962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:18.519862 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 14890
2026-09-27T07:00:18Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4178 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:18 dist-test-slave-p6hz krb5kdc[15488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:18 dist-test-slave-p6hz krb5kdc[15488](info): set up 2 sockets
Sep 27 07:00:18 dist-test-slave-p6hz krb5kdc[15488](info): commencing operation
krb5kdc: starting...
W20260927 07:00:19.519197 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.001s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:19 dist-test-slave-p6hz krb5kdc[15488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:19.696878 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:39251
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:39251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:20.013250 15501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:20.013566 15501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:20.013705 15501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:20.020946 15501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:20.021066 15501 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 07:00:20.021122 15501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:20.021205 15501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:20.021270 15501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:20.021350 15501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:20.031809 15501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:39251
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:39251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:20.033145 15501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:20.034387 15501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:20.045738 15506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:20.045709 15507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:20.045782 15509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:20.046491 15501 server_base.cc:1034] running on GCE node
I20260927 07:00:20.047250 15501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:20.048635 15501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:20.049841 15501 hybrid_clock.cc:648] HybridClock initialized: now 1790492420049783 us; error 38 us; skew 500 ppm
I20260927 07:00:20.050163 15501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:20 dist-test-slave-p6hz krb5kdc[15488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:20.053251 15501 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:20.054699 15501 webserver.cc:460] Webserver started at http://127.13.121.254:35353/ using document root <none> and password file <none>
I20260927 07:00:20.055426 15501 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:20.055527 15501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:20.055824 15501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:20.058722 15501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "f1eb5344764e4bf49590eb212d9bd28d"
format_stamp: "Formatted at 2026-09-27 07:00:20 on dist-test-slave-p6hz"
server_key: "9a91fd9ca965cf2c2604450062777a0e"
server_key_iv: "e5533d0c5a777c266762ecae6ae97be7"
server_key_version: "encryptionkey@0"
I20260927 07:00:20.059492 15501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "f1eb5344764e4bf49590eb212d9bd28d"
format_stamp: "Formatted at 2026-09-27 07:00:20 on dist-test-slave-p6hz"
server_key: "9a91fd9ca965cf2c2604450062777a0e"
server_key_iv: "e5533d0c5a777c266762ecae6ae97be7"
server_key_version: "encryptionkey@0"
I20260927 07:00:20.064805 15501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:00:20.068663 15516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:20.070237 15501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260927 07:00:20.070434 15501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "f1eb5344764e4bf49590eb212d9bd28d"
format_stamp: "Formatted at 2026-09-27 07:00:20 on dist-test-slave-p6hz"
server_key: "9a91fd9ca965cf2c2604450062777a0e"
server_key_iv: "e5533d0c5a777c266762ecae6ae97be7"
server_key_version: "encryptionkey@0"
I20260927 07:00:20.070676 15501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:20.096514 15501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:20.101382 15501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:20.101683 15501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:20.121812 15501 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:39251
I20260927 07:00:20.121831 15577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:39251 every 8 connection(s)
I20260927 07:00:20.123214 15501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:20.123981 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 15501
I20260927 07:00:20.124169 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:20.124953 13799 external_mini_cluster.cc:1428] Setting key b0bbd7b6834fe5060c2e6f2a485d5024
I20260927 07:00:20.127996 15578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 07:00:20 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:20.141144 15578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d: Bootstrap starting.
I20260927 07:00:20.145319 15578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:20.146606 15578 log.cc:824] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:20.149098 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:20.127045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55062 (local address 127.13.121.254:39251)
0927 07:00:20.127810 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:20.127821 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:20.127861 (+ 40us) server_negotiation.cc:407] Connection header received
0927 07:00:20.129452 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:20.129474 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:20.129867 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:20.130207 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:20.131906 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.133318 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.135123 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.135836 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.135974 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:20.140487 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:20.140499 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:20.140521 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:20.140574 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:20.144432 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:20.145212 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:20.145221 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:20.145230 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:20.145460 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:20.145856 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:20.145861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:20.145864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:20.146582 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:20.146864 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:20.147350 (+ 486us) server_negotiation.cc:299] Negotiation successful
0927 07:00:20.147804 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":341,"threads_started":1}
I20260927 07:00:20.150056 15578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d: No bootstrap required, opened a new log
I20260927 07:00:20.157590 15578 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb5344764e4bf49590eb212d9bd28d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39251 } }
I20260927 07:00:20.157951 15578 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:20.158021 15578 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1eb5344764e4bf49590eb212d9bd28d, State: Initialized, Role: FOLLOWER
I20260927 07:00:20.158573 15578 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb5344764e4bf49590eb212d9bd28d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39251 } }
I20260927 07:00:20.158855 15578 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:20.158994 15578 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:20.159188 15578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:20.161279 15578 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb5344764e4bf49590eb212d9bd28d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39251 } }
I20260927 07:00:20.161769 15578 leader_election.cc:302] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1eb5344764e4bf49590eb212d9bd28d; no voters:
I20260927 07:00:20.162326 15578 leader_election.cc:288] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:20.162446 15583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:20.163100 15583 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [term 1 LEADER]: Becoming Leader. State: Replica: f1eb5344764e4bf49590eb212d9bd28d, State: Running, Role: LEADER
I20260927 07:00:20.163683 15583 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb5344764e4bf49590eb212d9bd28d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39251 } }
I20260927 07:00:20.164292 15578 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:20.166776 15584 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1eb5344764e4bf49590eb212d9bd28d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb5344764e4bf49590eb212d9bd28d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39251 } } }
I20260927 07:00:20.167184 15584 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:20.167372 15585 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1eb5344764e4bf49590eb212d9bd28d. Latest consensus state: current_term: 1 leader_uuid: "f1eb5344764e4bf49590eb212d9bd28d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb5344764e4bf49590eb212d9bd28d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39251 } } }
I20260927 07:00:20.167757 15585 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:20.168833 15590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:20.174129 15590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:20.183379 15590 catalog_manager.cc:1347] Generated new cluster ID: 8e365eebfaf74fc396abd6b59afa0f83
I20260927 07:00:20.183512 15590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:20.191198 15590 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:20.192718 15590 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:20.199839 15590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d: Generated new TSK 0
I20260927 07:00:20.200714 15590 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:00:20.244910 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39251
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:20.568749 15604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:20.569037 15604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:20.569190 15604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:20.576526 15604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:20.576635 15604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:20.576740 15604 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 07:00:20.576800 15604 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:00:20.576896 15604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:20.586588 15604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.121.254:39251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:20.587859 15604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:20.589239 15604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:20.601550 15610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:20.601622 15609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:20.601593 15612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:20.602389 15604 server_base.cc:1034] running on GCE node
I20260927 07:00:20.602996 15604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:20.603907 15604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:20.605122 15604 hybrid_clock.cc:648] HybridClock initialized: now 1790492420605046 us; error 57 us; skew 500 ppm
I20260927 07:00:20.605428 15604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:20 dist-test-slave-p6hz krb5kdc[15488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:20.608359 15604 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:00:20.609766 15604 webserver.cc:460] Webserver started at http://127.13.121.193:33323/ using document root <none> and password file <none>
I20260927 07:00:20.610395 15604 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:20.610491 15604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:20.610738 15604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:20.613615 15604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "247d853cb67d4b99862f8598c8c11269"
format_stamp: "Formatted at 2026-09-27 07:00:20 on dist-test-slave-p6hz"
server_key: "55b4330924f8d3c4b04db6488a7c7ca2"
server_key_iv: "ad17816acd97caf039151a7040f45f78"
server_key_version: "encryptionkey@0"
I20260927 07:00:20.614308 15604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "247d853cb67d4b99862f8598c8c11269"
format_stamp: "Formatted at 2026-09-27 07:00:20 on dist-test-slave-p6hz"
server_key: "55b4330924f8d3c4b04db6488a7c7ca2"
server_key_iv: "ad17816acd97caf039151a7040f45f78"
server_key_version: "encryptionkey@0"
I20260927 07:00:20.619202 15604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:00:20.623185 15619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:20.624639 15604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260927 07:00:20.624802 15604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "247d853cb67d4b99862f8598c8c11269"
format_stamp: "Formatted at 2026-09-27 07:00:20 on dist-test-slave-p6hz"
server_key: "55b4330924f8d3c4b04db6488a7c7ca2"
server_key_iv: "ad17816acd97caf039151a7040f45f78"
server_key_version: "encryptionkey@0"
I20260927 07:00:20.624998 15604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:20.650837 15604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:20.654973 15604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:20.655279 15604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:20.657877 15604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:20.657976 15604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:20.658090 15604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:20.658169 15604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:00:20 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:20.679878 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:20.658497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:59241 (local address 127.13.121.254:39251)
0927 07:00:20.658966 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:20.658972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:20.660211 (+ 1239us) server_negotiation.cc:407] Connection header received
0927 07:00:20.662273 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:20.662279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:20.662365 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:20.662484 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:20.664527 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.665501 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.667912 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.668498 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.668606 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:20.673025 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:20.673030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:20.673033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:20.673067 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:20.675831 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:20.676874 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:20.676881 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:20.676884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:20.677100 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:20.677635 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:20.677640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:20.677644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:20.678023 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:20.678180 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:20.679243 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0927 07:00:20.679513 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":223,"threads_started":1}
I20260927 07:00:20.680884 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:20.659193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39251 (local address 127.13.121.193:59241)
0927 07:00:20.660029 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:20.660082 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:20.661997 (+ 1915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:20.662733 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:20.662741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:20.663459 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:20.664339 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:20.664372 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.665646 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.665651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:20.667782 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:20.667794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.668721 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.668728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:20.668940 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:20.669699 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:20.669743 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:20.672764 (+ 3021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:20.675983 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:20.675994 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:20.676014 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:20.676709 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:20.677245 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:20.677249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:20.677251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:20.677481 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:20.678179 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:20.678188 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:20.678720 (+ 532us) client_negotiation.cc:769] Sending connection context
0927 07:00:20.679100 (+ 380us) client_negotiation.cc:240] Negotiation successful
0927 07:00:20.679690 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":342,"threads_started":1}
I20260927 07:00:20.701138 15675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:20.686588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39251 (local address 127.13.121.193:52819)
0927 07:00:20.687250 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:20.687267 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:20.687398 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:20.687698 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:20.687702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:20.688133 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:20.688402 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:20.688411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.689394 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.689398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:20.690431 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:20.690444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.691598 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.691602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:20.691841 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:20.692571 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:20.692596 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:20.693668 (+ 1072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:20.697447 (+ 3779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:20.697453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:20.697457 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:20.698099 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:20.698620 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:20.698625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:20.698628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:20.698805 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:20.699757 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:20.699766 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:20.700010 (+ 244us) client_negotiation.cc:769] Sending connection context
0927 07:00:20.700154 (+ 144us) client_negotiation.cc:240] Negotiation successful
0927 07:00:20.700605 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":325,"threads_started":1}
I20260927 07:00:20.702340 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:20.686540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:52819 (local address 127.13.121.254:39251)
0927 07:00:20.686689 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:20.686693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:20.687346 (+ 653us) server_negotiation.cc:407] Connection header received
0927 07:00:20.687510 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:20.687514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:20.687574 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:20.687661 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:20.688540 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.689258 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.690571 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:20.691230 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:20.691357 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:20.693868 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:20.693876 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:20.693880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:20.693920 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:20.697268 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:20.698296 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:20.698302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:20.698307 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:20.698468 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:20.698959 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:20.698963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:20.698967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:20.699612 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:20.701587 (+ 1975us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:20.701719 (+ 132us) server_negotiation.cc:299] Negotiation successful
0927 07:00:20.702011 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 07:00:20.708909 15604 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:42191
I20260927 07:00:20.708940 15739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:42191 every 8 connection(s)
I20260927 07:00:20.710068 15604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:20.712059 15740 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39251
I20260927 07:00:20.712240 15740 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:20.712839 15740 heartbeater.cc:511] Master 127.13.121.254:39251 requested a full tablet report, sending...
I20260927 07:00:20.712877 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 15604
I20260927 07:00:20.713034 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:20.713690 13799 external_mini_cluster.cc:1428] Setting key 7f9e19230ed2f9ee9a679c62a0565688
I20260927 07:00:20.714789 15533 ts_manager.cc:194] Registered new tserver with Master: 247d853cb67d4b99862f8598c8c11269 (127.13.121.193:42191)
I20260927 07:00:20.716334 15533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:59241
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:00:20.750413 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39251
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:21.065184 15744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:21.065464 15744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:21.065589 15744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:21.073010 15744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:21.073136 15744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:21.073237 15744 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 07:00:21.073304 15744 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:00:21.073401 15744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:21.083367 15744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.121.254:39251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:21.084671 15744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:21.085980 15744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:21.099434 15749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:21.101176 15750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:21.101285 15752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:21.103199 15744 server_base.cc:1034] running on GCE node
I20260927 07:00:21.103983 15744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:21.105218 15744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:21.106434 15744 hybrid_clock.cc:648] HybridClock initialized: now 1790492421106372 us; error 66 us; skew 500 ppm
I20260927 07:00:21.106737 15744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:21 dist-test-slave-p6hz krb5kdc[15488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492421, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:21.110664 15744 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:00:21.112450 15744 webserver.cc:460] Webserver started at http://127.13.121.194:41735/ using document root <none> and password file <none>
I20260927 07:00:21.113350 15744 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:21.113489 15744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:21.113871 15744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:21.118076 15744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "0894a2e0553f4a72ab099c180ef78c8e"
format_stamp: "Formatted at 2026-09-27 07:00:21 on dist-test-slave-p6hz"
server_key: "1c2830450b53cf2643ceaaab1d635790"
server_key_iv: "1d16e3d006391777e0d0f3d3a0d6c715"
server_key_version: "encryptionkey@0"
I20260927 07:00:21.118997 15744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "0894a2e0553f4a72ab099c180ef78c8e"
format_stamp: "Formatted at 2026-09-27 07:00:21 on dist-test-slave-p6hz"
server_key: "1c2830450b53cf2643ceaaab1d635790"
server_key_iv: "1d16e3d006391777e0d0f3d3a0d6c715"
server_key_version: "encryptionkey@0"
I20260927 07:00:21.124925 15744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:00:21.128589 15759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:21.130098 15744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:21.130259 15744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "0894a2e0553f4a72ab099c180ef78c8e"
format_stamp: "Formatted at 2026-09-27 07:00:21 on dist-test-slave-p6hz"
server_key: "1c2830450b53cf2643ceaaab1d635790"
server_key_iv: "1d16e3d006391777e0d0f3d3a0d6c715"
server_key_version: "encryptionkey@0"
I20260927 07:00:21.130446 15744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:21.152825 15744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:21.156848 15744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:21.157112 15744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:21.159789 15744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:21.159907 15744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:21.160017 15744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:21.160094 15744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:00:21 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492421, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:21.183524 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.160395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:34071 (local address 127.13.121.254:39251)
0927 07:00:21.160607 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.160613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.162080 (+ 1467us) server_negotiation.cc:407] Connection header received
0927 07:00:21.164301 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.164307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.164399 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.164513 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:21.167332 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.168292 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.170792 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.171578 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.171738 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.176061 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:21.176070 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:21.176075 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:21.176128 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:21.179644 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.180581 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.180587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.180592 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.180777 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.181293 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.181298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.181302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.181690 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:21.181842 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.182878 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.183203 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260927 07:00:21.186169 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.161091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39251 (local address 127.13.121.194:34071)
0927 07:00:21.161903 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.161951 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.164006 (+ 2055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.165024 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.165038 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.166036 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:21.167071 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.167116 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.168488 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.168492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.170602 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.170627 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.171822 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.171827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.172037 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.172861 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:21.172886 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:21.175803 (+ 2917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:21.179862 (+ 4059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.179871 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.179888 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.180420 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.180931 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.180935 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.180938 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.181178 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.181848 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:21.181858 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:21.182421 (+ 563us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.183758 (+ 1337us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.184393 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":331,"threads_started":1}
I20260927 07:00:21.207510 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.192429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39251 (local address 127.13.121.194:56143)
0927 07:00:21.193071 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.193093 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.193272 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.193589 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.193594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.194251 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:21.194649 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.194663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.195961 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.195967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.197970 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.197989 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.199060 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.199066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.199248 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.199954 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:21.199981 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:21.201126 (+ 1145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:21.204521 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.204527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.204532 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.205137 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.205548 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.205554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.205558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.205716 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.206246 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:21.206251 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:21.206467 (+ 216us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.206626 (+ 159us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.207004 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":313,"threads_started":1}
I20260927 07:00:21.207717 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.192443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:56143 (local address 127.13.121.254:39251)
0927 07:00:21.192676 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.192683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.193203 (+ 520us) server_negotiation.cc:407] Connection header received
0927 07:00:21.193385 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.193388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.193463 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.193545 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:21.194831 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.195815 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.198212 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.198853 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.198987 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.201374 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:21.201380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:21.201383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:21.201426 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:21.204351 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.205280 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.205285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.205287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.205437 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.205811 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.205814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.205816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.206151 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:21.206260 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.206795 (+ 535us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.207436 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:00:21.215503 15744 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:41769
I20260927 07:00:21.215538 15878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:41769 every 8 connection(s)
I20260927 07:00:21.216722 15744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:21.217252 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 15744
I20260927 07:00:21.217417 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:21.217844 13799 external_mini_cluster.cc:1428] Setting key 36021a6f2179e50c69e4808137497dba
I20260927 07:00:21.219099 15879 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39251
I20260927 07:00:21.219336 15879 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:21.220059 15879 heartbeater.cc:511] Master 127.13.121.254:39251 requested a full tablet report, sending...
I20260927 07:00:21.221432 15533 ts_manager.cc:194] Registered new tserver with Master: 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769)
I20260927 07:00:21.222436 15533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:34071
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:00:21.257735 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39251
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:21.601649 15883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:21.601927 15883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:21.602033 15883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:21.609737 15883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:21.609874 15883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:21.609965 15883 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 07:00:21.610018 15883 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:00:21.610086 15883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:21.619936 15883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.121.254:39251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:21.621243 15883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:21.622540 15883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:21.636049 15889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:21.636128 15891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:21.636078 15888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:21.636900 15883 server_base.cc:1034] running on GCE node
I20260927 07:00:21.637476 15883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:21.638445 15883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:21.639726 15883 hybrid_clock.cc:648] HybridClock initialized: now 1790492421639663 us; error 49 us; skew 500 ppm
I20260927 07:00:21.640103 15883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:21 dist-test-slave-p6hz krb5kdc[15488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492421, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:21.643520 15883 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:00:21.645018 15883 webserver.cc:460] Webserver started at http://127.13.121.195:43011/ using document root <none> and password file <none>
I20260927 07:00:21.645758 15883 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:21.645879 15883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:21.646196 15883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:21.649274 15883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "5fc855808dda4646af609a92adc265ef"
format_stamp: "Formatted at 2026-09-27 07:00:21 on dist-test-slave-p6hz"
server_key: "02f7aabc1be37283a370496e400b7980"
server_key_iv: "5908e502db2314988af43ce4c09d9d0a"
server_key_version: "encryptionkey@0"
I20260927 07:00:21.650058 15883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "5fc855808dda4646af609a92adc265ef"
format_stamp: "Formatted at 2026-09-27 07:00:21 on dist-test-slave-p6hz"
server_key: "02f7aabc1be37283a370496e400b7980"
server_key_iv: "5908e502db2314988af43ce4c09d9d0a"
server_key_version: "encryptionkey@0"
I20260927 07:00:21.655510 15883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 07:00:21.659317 15898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:21.660815 15883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:21.660982 15883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "5fc855808dda4646af609a92adc265ef"
format_stamp: "Formatted at 2026-09-27 07:00:21 on dist-test-slave-p6hz"
server_key: "02f7aabc1be37283a370496e400b7980"
server_key_iv: "5908e502db2314988af43ce4c09d9d0a"
server_key_version: "encryptionkey@0"
I20260927 07:00:21.661218 15883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:21.687460 15883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:21.691812 15883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:21.692101 15883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:21.694751 15883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:21.694859 15883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:21.694964 15883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:21.695039 15883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:00:21 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492421, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:21.719003 15740 heartbeater.cc:503] Master 127.13.121.254:39251 was elected leader, sending a full tablet report...
I20260927 07:00:21.720687 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.695472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:33109 (local address 127.13.121.254:39251)
0927 07:00:21.695677 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.695685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.697193 (+ 1508us) server_negotiation.cc:407] Connection header received
0927 07:00:21.699983 (+ 2790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.699989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.700084 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.700204 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:21.702878 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.703913 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.706943 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.707510 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.707665 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.713091 (+ 5426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:21.713098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:21.713102 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:21.713148 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:21.716894 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.717915 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.717921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.717925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.718057 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.718482 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.718486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.718488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.718756 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:21.718876 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.720034 (+ 1158us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.720308 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:00:21.722261 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.696085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39251 (local address 127.13.121.195:33109)
0927 07:00:21.696993 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.697048 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.699590 (+ 2542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.700505 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.700518 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.701506 (+ 988us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:21.702601 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.702650 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.704112 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.704119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.706711 (+ 2592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.706736 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.707978 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.707985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.708218 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.709138 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:21.709171 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:21.712745 (+ 3574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:21.717117 (+ 4372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.717137 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.717152 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.717782 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.718177 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.718181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.718184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.718374 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.718887 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:21.718895 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:21.719468 (+ 573us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.720470 (+ 1002us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.721205 (+ 735us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":336,"threads_started":1}
I20260927 07:00:21.740286 15883 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:33705
I20260927 07:00:21.740315 16017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:33705 every 8 connection(s)
I20260927 07:00:21.741638 15883 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:21.742966 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.729124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:46615 (local address 127.13.121.254:39251)
0927 07:00:21.729322 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.729328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.729907 (+ 579us) server_negotiation.cc:407] Connection header received
0927 07:00:21.730108 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.730112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.730178 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.730271 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:21.731759 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.732461 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.734517 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.734989 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.735081 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.737751 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:21.737758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:21.737763 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:21.737802 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:21.740418 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.741078 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.741085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.741088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.741216 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.741536 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.741542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.741546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.741849 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:21.741975 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.742387 (+ 412us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.742601 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 07:00:21.743095 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.729136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39251 (local address 127.13.121.195:46615)
0927 07:00:21.729812 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.729831 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.729981 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.730340 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.730345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.731071 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:21.731582 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.731599 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.732605 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.732611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.734358 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.734374 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.735191 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.735198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.735390 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.736293 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:21.736326 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:21.737520 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:21.740550 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.740554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.740558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.740968 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.741319 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.741324 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.741328 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.741442 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.741952 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:21.741957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:21.742147 (+ 190us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.742278 (+ 131us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.742644 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":329,"threads_started":1}
I20260927 07:00:21.743803 16018 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39251
I20260927 07:00:21.744071 16018 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:21.745016 16018 heartbeater.cc:511] Master 127.13.121.254:39251 requested a full tablet report, sending...
I20260927 07:00:21.745141 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 15883
I20260927 07:00:21.745296 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:21.745779 13799 external_mini_cluster.cc:1428] Setting key 28dd809631c958a9895a63446a2153aa
I20260927 07:00:21.746963 15532 ts_manager.cc:194] Registered new tserver with Master: 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705)
I20260927 07:00:21.748311 15532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:33109
I20260927 07:00:21.750918 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:00:21 dist-test-slave-p6hz krb5kdc[15488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 07:00:21 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:21.794363 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.780752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.13.121.254:39251)
0927 07:00:21.780959 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.780966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.781209 (+ 243us) server_negotiation.cc:407] Connection header received
0927 07:00:21.781379 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.781383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.781444 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.781527 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:21.782983 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.783803 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.785043 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.785456 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.785568 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.789264 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:21.789269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:21.789273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:21.789310 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:21.791848 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.792475 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.792480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.792482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.792603 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.792952 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.792957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.792960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.793248 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:21.793480 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.793780 (+ 300us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.794020 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:00:21.798700 15533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:00:21.805938 15533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:21.830724 16033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.818513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42191 (local address 127.0.0.1:53428)
0927 07:00:21.820147 (+ 1634us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.820194 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.820378 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.821904 (+ 1526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.821913 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.821950 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:21.822404 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.822416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.824817 (+ 2401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.824823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.827403 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.827421 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.829623 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.829631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.829899 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.829967 (+ 68us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.830103 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.830316 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1282,"threads_started":1}
I20260927 07:00:21.832137 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.818759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.13.121.193:42191)
0927 07:00:21.820885 (+ 2126us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.820894 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.820921 (+ 27us) server_negotiation.cc:407] Connection header received
0927 07:00:21.821030 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.821038 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.821305 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.822648 (+ 1343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:21.822852 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.824580 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.827641 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.829344 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.831317 (+ 1973us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.831504 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.831574 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.831743 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1992,"thread_start_us":1883,"threads_started":1}
I20260927 07:00:21.836501 15671 tablet_service.cc:1467] Processing CreateTablet for tablet 079d6e51cdb74be2b4102d95ebae4489 (DEFAULT_TABLE table=test-table [id=bb7f454ba83c4bf894fc62fec331a669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:21.838119 15671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079d6e51cdb74be2b4102d95ebae4489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:21.838882 16033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.818736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33705 (local address 127.0.0.1:51534)
0927 07:00:21.830967 (+ 12231us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.830994 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.831098 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.833014 (+ 1916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.833019 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.833046 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:21.833207 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.833217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.835164 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.835170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.836862 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.836873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.838124 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.838131 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.838311 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.838343 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.838431 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.838582 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12114,"thread_start_us":921,"threads_started":1}
I20260927 07:00:21.839622 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.819379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.13.121.195:33705)
0927 07:00:21.832401 (+ 13022us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.832410 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.832447 (+ 37us) server_negotiation.cc:407] Connection header received
0927 07:00:21.832556 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.832564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.832807 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.833362 (+ 555us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:21.833673 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.834980 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.837007 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.837959 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.838149 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.838326 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.838481 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.838638 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12848,"thread_start_us":12670,"threads_started":1}
I20260927 07:00:21.840790 16035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.821532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:41769 (local address 127.0.0.1:35340)
0927 07:00:21.831570 (+ 10038us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.831590 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.831724 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.833335 (+ 1611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.833339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.833363 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:21.833666 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.833676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.836484 (+ 2808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.836489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.838477 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.838491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.840237 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.840241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.840364 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.840388 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.840459 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.840563 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9899,"thread_start_us":9219,"threads_started":1}
I20260927 07:00:21.840968 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.821864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35340 (local address 127.13.121.194:41769)
0927 07:00:21.832813 (+ 10949us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.832821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.832847 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:00:21.832939 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.832946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.833160 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.833339 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:21.834699 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.836340 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.838669 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.840038 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.840215 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.840376 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.840545 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.840703 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10781,"thread_start_us":11058,"threads_started":1}
I20260927 07:00:21.844527 15809 tablet_service.cc:1467] Processing CreateTablet for tablet 079d6e51cdb74be2b4102d95ebae4489 (DEFAULT_TABLE table=test-table [id=bb7f454ba83c4bf894fc62fec331a669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:21.846081 15809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079d6e51cdb74be2b4102d95ebae4489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:21.847980 15948 tablet_service.cc:1467] Processing CreateTablet for tablet 079d6e51cdb74be2b4102d95ebae4489 (DEFAULT_TABLE table=test-table [id=bb7f454ba83c4bf894fc62fec331a669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:21.849407 15948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079d6e51cdb74be2b4102d95ebae4489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:21.851487 16037 tablet_bootstrap.cc:492] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: Bootstrap starting.
I20260927 07:00:21.854173 16037 tablet_bootstrap.cc:654] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:21.855194 16037 log.cc:824] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:21.857909 16037 tablet_bootstrap.cc:492] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: No bootstrap required, opened a new log
I20260927 07:00:21.858202 16037 ts_tablet_manager.cc:1397] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260927 07:00:21.860671 16039 tablet_bootstrap.cc:492] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: Bootstrap starting.
I20260927 07:00:21.863726 16040 tablet_bootstrap.cc:492] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: Bootstrap starting.
I20260927 07:00:21.864637 16039 tablet_bootstrap.cc:654] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:21.866087 16039 log.cc:824] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:21.867420 16040 tablet_bootstrap.cc:654] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:21.868525 16040 log.cc:824] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:21.868371 16037 raft_consensus.cc:348] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } }
I20260927 07:00:21.868883 16037 raft_consensus.cc:374] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:21.868986 16037 raft_consensus.cc:729] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247d853cb67d4b99862f8598c8c11269, State: Initialized, Role: FOLLOWER
I20260927 07:00:21.869869 16037 consensus_queue.cc:260] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } }
I20260927 07:00:21.872037 16037 ts_tablet_manager.cc:1428] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260927 07:00:21.875422 16040 tablet_bootstrap.cc:492] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: No bootstrap required, opened a new log
I20260927 07:00:21.875746 16040 ts_tablet_manager.cc:1397] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260927 07:00:21.876014 16039 tablet_bootstrap.cc:492] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: No bootstrap required, opened a new log
I20260927 07:00:21.876307 16039 ts_tablet_manager.cc:1397] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260927 07:00:21.877036 16043 raft_consensus.cc:482] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:21.878681 16043 raft_consensus.cc:504] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } }
I20260927 07:00:21.885619 16040 raft_consensus.cc:348] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } }
I20260927 07:00:21.886207 16040 raft_consensus.cc:374] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:21.886317 16040 raft_consensus.cc:729] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc855808dda4646af609a92adc265ef, State: Initialized, Role: FOLLOWER
I20260927 07:00:21.887321 16043 leader_election.cc:288] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705), 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769)
I20260927 07:00:21.887272 16040 consensus_queue.cc:260] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } }
I20260927 07:00:21.887508 16039 raft_consensus.cc:348] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } }
I20260927 07:00:21.888131 16039 raft_consensus.cc:374] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:21.888288 16039 raft_consensus.cc:729] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0894a2e0553f4a72ab099c180ef78c8e, State: Initialized, Role: FOLLOWER
I20260927 07:00:21.889165 16039 consensus_queue.cc:260] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } }
I20260927 07:00:21.895855 16040 ts_tablet_manager.cc:1428] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: Time spent starting tablet: real 0.020s user 0.019s sys 0.000s
I20260927 07:00:21.896299 16018 heartbeater.cc:503] Master 127.13.121.254:39251 was elected leader, sending a full tablet report...
I20260927 07:00:21.903812 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.881635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:38845 (local address 127.13.121.195:33705)
0927 07:00:21.881822 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.881828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.894797 (+ 12969us) server_negotiation.cc:407] Connection header received
0927 07:00:21.895462 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.895468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.895614 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.895684 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:21.896571 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.897801 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.901602 (+ 3801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.902704 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.902810 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.902863 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.903418 (+ 555us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.903568 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:00:21.903940 15879 heartbeater.cc:503] Master 127.13.121.254:39251 was elected leader, sending a full tablet report...
I20260927 07:00:21.904556 16045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.881171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33705 (local address 127.13.121.193:38845)
0927 07:00:21.894215 (+ 13044us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.894244 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.895246 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.895916 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.895921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.895953 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:21.896387 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.896398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.898012 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.898019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.900575 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.900596 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.903042 (+ 2446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.903047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.903223 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.903259 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.903988 (+ 729us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.904147 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12864,"thread_start_us":19698,"threads_started":1}
I20260927 07:00:21.904748 16039 ts_tablet_manager.cc:1428] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: Time spent starting tablet: real 0.028s user 0.016s sys 0.012s
I20260927 07:00:21.905813 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079d6e51cdb74be2b4102d95ebae4489" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef" is_pre_election: true
I20260927 07:00:21.906373 15968 raft_consensus.cc:2454] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 0.
I20260927 07:00:21.907099 15623 leader_election.cc:302] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 247d853cb67d4b99862f8598c8c11269, 5fc855808dda4646af609a92adc265ef; no voters:
I20260927 07:00:21.908121 16043 raft_consensus.cc:2781] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:21.908280 16043 raft_consensus.cc:482] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:21.908389 16043 raft_consensus.cc:3037] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:21.910504 16043 raft_consensus.cc:504] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } }
I20260927 07:00:21.911417 16043 leader_election.cc:288] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 election: Requested vote from peers 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705), 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769)
I20260927 07:00:21.911599 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.881721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:52687 (local address 127.13.121.194:41769)
0927 07:00:21.881907 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.881913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.901334 (+ 19421us) server_negotiation.cc:407] Connection header received
0927 07:00:21.901634 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.901641 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.901725 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.901790 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:21.902679 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.903874 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.906004 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.906965 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.907117 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.907210 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.911130 (+ 3920us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.911324 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 07:00:21.911703 16044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.881251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:41769 (local address 127.13.121.193:52687)
0927 07:00:21.901216 (+ 19965us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.901242 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.901457 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.901990 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.901995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.902022 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:21.902467 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.902480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.904062 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.904067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.905859 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.905871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.907436 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.907443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.910860 (+ 3417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.910916 (+ 56us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.911041 (+ 125us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.911233 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19782,"thread_start_us":13562,"threads_started":1}
I20260927 07:00:21.911767 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079d6e51cdb74be2b4102d95ebae4489" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef"
I20260927 07:00:21.911998 15968 raft_consensus.cc:3037] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:21.912746 15833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079d6e51cdb74be2b4102d95ebae4489" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0894a2e0553f4a72ab099c180ef78c8e"
I20260927 07:00:21.912772 15834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079d6e51cdb74be2b4102d95ebae4489" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0894a2e0553f4a72ab099c180ef78c8e" is_pre_election: true
I20260927 07:00:21.913218 15834 raft_consensus.cc:2454] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 0.
I20260927 07:00:21.914698 15968 raft_consensus.cc:2454] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 1.
I20260927 07:00:21.915195 15623 leader_election.cc:302] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 247d853cb67d4b99862f8598c8c11269, 5fc855808dda4646af609a92adc265ef; no voters: 0894a2e0553f4a72ab099c180ef78c8e
I20260927 07:00:21.915524 16043 raft_consensus.cc:2781] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:21.916366 16043 raft_consensus.cc:686] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 1 LEADER]: Becoming Leader. State: Replica: 247d853cb67d4b99862f8598c8c11269, State: Running, Role: LEADER
I20260927 07:00:21.916867 16043 consensus_queue.cc:237] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } }
I20260927 07:00:21.921757 15533 catalog_manager.cc:5495] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 reported cstate change: term changed from 0 to 1, leader changed from <none> to 247d853cb67d4b99862f8598c8c11269 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } health_report { overall_health: UNKNOWN } } }
W20260927 07:00:21.962378 15741 tablet.cc:2369] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:00:21.969681 15880 tablet.cc:2369] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 07:00:21 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492420, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:21.978520 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.961573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.13.121.254:39251)
0927 07:00:21.961759 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:21.961763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:21.962267 (+ 504us) server_negotiation.cc:407] Connection header received
0927 07:00:21.962427 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:21.962431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:21.962509 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:21.962606 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:21.963762 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.964509 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.965832 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.966250 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.966352 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.973142 (+ 6790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:21.973147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:21.973150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:21.973181 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:21.975797 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.976539 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.976544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.976547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.976692 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:21.977034 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:21.977039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:21.977041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:21.977317 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:21.977454 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:21.977909 (+ 455us) server_negotiation.cc:299] Negotiation successful
0927 07:00:21.978113 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 07:00:21.978627 16055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:21.961589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39251 (local address 127.0.0.1:55080)
0927 07:00:21.962163 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:21.962178 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:21.962320 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:21.962658 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:21.962662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:21.963355 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:21.963625 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.963635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.964635 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.964639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.965710 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:21.965723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:21.966422 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:21.966428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:21.966585 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:21.970679 (+ 4094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:21.970700 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:21.972935 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:21.975937 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.975953 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.975962 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.976413 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.976795 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:21.976800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:21.976804 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:21.976929 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:21.977448 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:21.977457 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:21.977663 (+ 206us) client_negotiation.cc:769] Sending connection context
0927 07:00:21.977805 (+ 142us) client_negotiation.cc:240] Negotiation successful
0927 07:00:21.978248 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":137,"thread_start_us":309,"threads_started":1}
I20260927 07:00:21.983942 15533 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:55080:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 07:00:21.984952 15533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:21.991554 15809 tablet_service.cc:1467] Processing CreateTablet for tablet 3d609e8ccf144e75ad82e58495df1c21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=68870a39022c4089a6e483da0ed456a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:00:21.991712 15948 tablet_service.cc:1467] Processing CreateTablet for tablet 3d609e8ccf144e75ad82e58495df1c21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=68870a39022c4089a6e483da0ed456a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:00:21.992542 15809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d609e8ccf144e75ad82e58495df1c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:21.992547 15948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d609e8ccf144e75ad82e58495df1c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:21.995579 15671 tablet_service.cc:1467] Processing CreateTablet for tablet 3d609e8ccf144e75ad82e58495df1c21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=68870a39022c4089a6e483da0ed456a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:00:21.996368 15671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d609e8ccf144e75ad82e58495df1c21. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 07:00:21.999208 16019 tablet.cc:2369] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:00:21.999586 16040 tablet_bootstrap.cc:492] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: Bootstrap starting.
I20260927 07:00:21.999617 16039 tablet_bootstrap.cc:492] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: Bootstrap starting.
I20260927 07:00:22.001101 16040 tablet_bootstrap.cc:654] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:22.001304 16039 tablet_bootstrap.cc:654] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:22.003060 16040 tablet_bootstrap.cc:492] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: No bootstrap required, opened a new log
I20260927 07:00:22.003221 16040 ts_tablet_manager.cc:1397] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260927 07:00:22.003816 16040 raft_consensus.cc:348] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.004000 16040 raft_consensus.cc:374] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:22.004088 16040 raft_consensus.cc:729] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc855808dda4646af609a92adc265ef, State: Initialized, Role: FOLLOWER
I20260927 07:00:22.004294 16040 consensus_queue.cc:260] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.004698 16046 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.004987 16037 tablet_bootstrap.cc:492] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: Bootstrap starting.
I20260927 07:00:22.005045 16046 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:22.005473 16040 ts_tablet_manager.cc:1428] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:00:22.006386 16037 tablet_bootstrap.cc:654] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:22.007675 16039 tablet_bootstrap.cc:492] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: No bootstrap required, opened a new log
I20260927 07:00:22.007808 16039 ts_tablet_manager.cc:1397] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.001s
I20260927 07:00:22.008410 16039 raft_consensus.cc:348] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.008606 16039 raft_consensus.cc:374] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:22.008699 16039 raft_consensus.cc:729] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0894a2e0553f4a72ab099c180ef78c8e, State: Initialized, Role: FOLLOWER
I20260927 07:00:22.008908 16039 consensus_queue.cc:260] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.009507 16039 ts_tablet_manager.cc:1428] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260927 07:00:22.009331 16047 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.009763 16047 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:22.011245 16037 tablet_bootstrap.cc:492] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: No bootstrap required, opened a new log
I20260927 07:00:22.011384 16037 ts_tablet_manager.cc:1397] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260927 07:00:22.012153 16037 raft_consensus.cc:348] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.012408 16037 raft_consensus.cc:374] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:22.012516 16037 raft_consensus.cc:729] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247d853cb67d4b99862f8598c8c11269, State: Initialized, Role: FOLLOWER
I20260927 07:00:22.012741 16037 consensus_queue.cc:260] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.013264 16037 ts_tablet_manager.cc:1428] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260927 07:00:22.013188 16043 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.013485 16043 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:22.112229 16043 raft_consensus.cc:482] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:22.112443 16043 raft_consensus.cc:504] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.113070 16043 leader_election.cc:288] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769), 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705)
I20260927 07:00:22.113642 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d609e8ccf144e75ad82e58495df1c21" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef" is_pre_election: true
I20260927 07:00:22.113665 15834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d609e8ccf144e75ad82e58495df1c21" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0894a2e0553f4a72ab099c180ef78c8e" is_pre_election: true
I20260927 07:00:22.113873 15834 raft_consensus.cc:2454] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 0.
I20260927 07:00:22.113888 15968 raft_consensus.cc:2454] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 0.
I20260927 07:00:22.114274 15623 leader_election.cc:302] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 247d853cb67d4b99862f8598c8c11269, 5fc855808dda4646af609a92adc265ef; no voters:
I20260927 07:00:22.114567 16043 raft_consensus.cc:2781] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:22.114650 16043 raft_consensus.cc:482] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:22.114746 16043 raft_consensus.cc:3037] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:22.117178 16043 raft_consensus.cc:504] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.117753 16043 leader_election.cc:288] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 election: Requested vote from peers 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769), 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705)
I20260927 07:00:22.118258 15834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d609e8ccf144e75ad82e58495df1c21" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0894a2e0553f4a72ab099c180ef78c8e"
I20260927 07:00:22.118251 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d609e8ccf144e75ad82e58495df1c21" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef"
I20260927 07:00:22.118456 15968 raft_consensus.cc:3037] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:22.118453 15834 raft_consensus.cc:3037] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:22.120468 15968 raft_consensus.cc:2454] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 1.
I20260927 07:00:22.120682 15834 raft_consensus.cc:2454] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 1.
I20260927 07:00:22.120919 15623 leader_election.cc:302] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 247d853cb67d4b99862f8598c8c11269, 5fc855808dda4646af609a92adc265ef; no voters:
I20260927 07:00:22.121274 16043 raft_consensus.cc:2781] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:22.121474 16043 raft_consensus.cc:686] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [term 1 LEADER]: Becoming Leader. State: Replica: 247d853cb67d4b99862f8598c8c11269, State: Running, Role: LEADER
I20260927 07:00:22.121737 16043 consensus_queue.cc:237] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.123418 16048 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: TxnStatusTablet state changed. Reason: New leader 247d853cb67d4b99862f8598c8c11269. Latest consensus state: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.123777 16048 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:00:22.124476 16060 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 07:00:22.124756 16060 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 07:00:22.124970 15532 catalog_manager.cc:5495] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 reported cstate change: term changed from 0 to 1, leader changed from <none> to 247d853cb67d4b99862f8598c8c11269 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } health_report { overall_health: UNKNOWN } } }
I20260927 07:00:22.161497 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.154866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.13.121.193:42191)
0927 07:00:22.155069 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:22.155075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:22.155285 (+ 210us) server_negotiation.cc:407] Connection header received
0927 07:00:22.155560 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:22.155565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:22.155651 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:22.155767 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:22.156131 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.156885 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.159010 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.159545 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.159664 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.160315 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.160909 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.161065 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:22.161202 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 07:00:22.161275 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:00:22.161643 16055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.154947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42191 (local address 127.0.0.1:53444)
0927 07:00:22.155203 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:22.155219 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:22.155364 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:22.155823 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:22.155827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:22.155840 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:00:22.156000 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.156009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.157045 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.157051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.158771 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.158798 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.159718 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.159725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.159962 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.160062 (+ 100us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.161057 (+ 995us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.161083 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 07:00:22.161167 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 07:00:22.161337 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260927 07:00:22.169628 15834 raft_consensus.cc:1261] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Refusing update from remote peer 247d853cb67d4b99862f8598c8c11269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:22.169764 15968 raft_consensus.cc:1261] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Refusing update from remote peer 247d853cb67d4b99862f8598c8c11269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:22.170568 16048 consensus_queue.cc:1035] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:22.170923 16043 consensus_queue.cc:1035] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:22.189081 16046 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: TxnStatusTablet state changed. Reason: New leader 247d853cb67d4b99862f8598c8c11269. Latest consensus state: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.189384 16046 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:22.191052 16063 mvcc.cc:205] Tried to move back new op lower bound from 7333856961196957696 to 7333856961012867072. Current Snapshot: MvccSnapshot[applied={T|T < 7333856961196957696}]
I20260927 07:00:22.193851 16065 mvcc.cc:205] Tried to move back new op lower bound from 7333856961196957696 to 7333856961012867072. Current Snapshot: MvccSnapshot[applied={T|T < 7333856961196957696}]
I20260927 07:00:22.200177 16046 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.200452 16046 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 5fc855808dda4646af609a92adc265ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:22.200860 16043 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.201175 16043 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:00:22.209085 16047 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: TxnStatusTablet state changed. Reason: New leader 247d853cb67d4b99862f8598c8c11269. Latest consensus state: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.209373 16047 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:22.210609 16048 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.210888 16048 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 247d853cb67d4b99862f8598c8c11269: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:00:22.224974 16047 tablet_replica.cc:439] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } } }
I20260927 07:00:22.225238 16047 tablet_replica.cc:442] T 3d609e8ccf144e75ad82e58495df1c21 P 0894a2e0553f4a72ab099c180ef78c8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:00:22.231494 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.225178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.13.121.193:42191)
0927 07:00:22.225346 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:22.225351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:22.225369 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:00:22.225436 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:22.225441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:22.225531 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:22.225612 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:22.226154 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.227275 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.229322 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.230022 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.230151 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.230672 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.230829 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.230925 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:22.231069 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 07:00:22.231180 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 07:00:22.250495 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.244355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:47385 (local address 127.13.121.193:42191)
0927 07:00:22.244732 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:22.244746 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:22.245086 (+ 340us) server_negotiation.cc:407] Connection header received
0927 07:00:22.245243 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:22.245248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:22.245327 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:22.245434 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:22.245929 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.247027 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.248513 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.249237 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.249359 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.249760 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.249891 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.249972 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:22.250096 (+ 124us) server_negotiation.cc:299] Negotiation successful
0927 07:00:22.250173 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260927 07:00:22.250888 16075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.244369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42191 (local address 127.13.121.193:47385)
0927 07:00:22.244984 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:22.245005 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:22.245131 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:22.245442 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:22.245445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:22.245454 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:00:22.245780 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.245789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.247194 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.247200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.248382 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.248398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.249404 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.249408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.249516 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.249576 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.249985 (+ 409us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.249997 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 07:00:22.250054 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 07:00:22.250190 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":311,"threads_started":1}
I20260927 07:00:22.259639 15834 raft_consensus.cc:3037] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:22.259924 15968 raft_consensus.cc:1261] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Refusing update from remote peer 247d853cb67d4b99862f8598c8c11269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:22.260443 16043 consensus_queue.cc:1035] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:22.262002 15834 raft_consensus.cc:1261] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Refusing update from remote peer 247d853cb67d4b99862f8598c8c11269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:22.262537 16043 consensus_queue.cc:1035] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:22.272296 16064 mvcc.cc:205] Tried to move back new op lower bound from 7333856961571373056 to 7333856960175226880. Current Snapshot: MvccSnapshot[applied={T|T < 7333856961571373056}]
I20260927 07:00:22.332060 15533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:22.332291 15533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:22.335911 15533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d: Sending DeleteTablet for 3 replicas of tablet 079d6e51cdb74be2b4102d95ebae4489
I20260927 07:00:22.336928 15671 tablet_service.cc:1514] Processing DeleteTablet for tablet 079d6e51cdb74be2b4102d95ebae4489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:22 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:53428
I20260927 07:00:22.337005 15948 tablet_service.cc:1514] Processing DeleteTablet for tablet 079d6e51cdb74be2b4102d95ebae4489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:22 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:51534
I20260927 07:00:22.337059 15809 tablet_service.cc:1514] Processing DeleteTablet for tablet 079d6e51cdb74be2b4102d95ebae4489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:22 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35340
I20260927 07:00:22.341331 15533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 07:00:22.342303 16079 tablet_replica.cc:330] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: stopping tablet replica
W20260927 07:00:22.342514 15533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:22.342818 16079 raft_consensus.cc:2229] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:22.344627 16079 raft_consensus.cc:2258] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:22.347386 16079 ts_tablet_manager.cc:1905] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:22.349711 16081 tablet_replica.cc:330] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: stopping tablet replica
I20260927 07:00:22.350148 16081 raft_consensus.cc:2229] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:22.350504 16081 raft_consensus.cc:2258] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:22.350862 15669 tablet_service.cc:1467] Processing CreateTablet for tablet f0fe38ab5746460e9b117fb1fbc42cec (DEFAULT_TABLE table=test-table [id=2c93cc2321a44c11927600d04d9d2078]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:22.351327 16080 tablet_replica.cc:330] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: stopping tablet replica
I20260927 07:00:22.351501 15669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0fe38ab5746460e9b117fb1fbc42cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:22.351768 16080 raft_consensus.cc:2229] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:00:22.352356 16080 raft_consensus.cc:2258] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:22.352797 16081 ts_tablet_manager.cc:1905] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:22.354417 16080 ts_tablet_manager.cc:1905] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:22.356400 15948 tablet_service.cc:1467] Processing CreateTablet for tablet f0fe38ab5746460e9b117fb1fbc42cec (DEFAULT_TABLE table=test-table [id=2c93cc2321a44c11927600d04d9d2078]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:22.356598 15809 tablet_service.cc:1467] Processing CreateTablet for tablet f0fe38ab5746460e9b117fb1fbc42cec (DEFAULT_TABLE table=test-table [id=2c93cc2321a44c11927600d04d9d2078]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:22.357009 15948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0fe38ab5746460e9b117fb1fbc42cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:22.357048 15809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0fe38ab5746460e9b117fb1fbc42cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:22.359405 16079 ts_tablet_manager.cc:1918] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:00:22.359557 16079 log.cc:1196] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/079d6e51cdb74be2b4102d95ebae4489
I20260927 07:00:22.360020 16079 ts_tablet_manager.cc:1939] T 079d6e51cdb74be2b4102d95ebae4489 P 5fc855808dda4646af609a92adc265ef: Deleting consensus metadata
I20260927 07:00:22.361917 15520 catalog_manager.cc:4841] TS 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705): tablet 079d6e51cdb74be2b4102d95ebae4489 (table test-table [id=bb7f454ba83c4bf894fc62fec331a669]) successfully deleted
I20260927 07:00:22.363306 16039 tablet_bootstrap.cc:492] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: Bootstrap starting.
I20260927 07:00:22.365121 16039 tablet_bootstrap.cc:654] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:22.365288 16037 tablet_bootstrap.cc:492] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: Bootstrap starting.
I20260927 07:00:22.366977 16037 tablet_bootstrap.cc:654] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:22.367489 16081 ts_tablet_manager.cc:1918] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:00:22.367659 16081 log.cc:1196] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/079d6e51cdb74be2b4102d95ebae4489
I20260927 07:00:22.368166 16081 ts_tablet_manager.cc:1939] T 079d6e51cdb74be2b4102d95ebae4489 P 0894a2e0553f4a72ab099c180ef78c8e: Deleting consensus metadata
I20260927 07:00:22.370002 15518 catalog_manager.cc:4841] TS 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769): tablet 079d6e51cdb74be2b4102d95ebae4489 (table test-table [id=bb7f454ba83c4bf894fc62fec331a669]) successfully deleted
I20260927 07:00:22.373757 16040 tablet_bootstrap.cc:492] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: Bootstrap starting.
I20260927 07:00:22.375459 16040 tablet_bootstrap.cc:654] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:22.376143 16039 tablet_bootstrap.cc:492] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: No bootstrap required, opened a new log
I20260927 07:00:22.376310 16039 ts_tablet_manager.cc:1397] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.005s
I20260927 07:00:22.377048 16039 raft_consensus.cc:348] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.377321 16039 raft_consensus.cc:374] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:22.378574 16039 raft_consensus.cc:729] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0894a2e0553f4a72ab099c180ef78c8e, State: Initialized, Role: FOLLOWER
I20260927 07:00:22.378882 16039 consensus_queue.cc:260] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.380077 16037 tablet_bootstrap.cc:492] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: No bootstrap required, opened a new log
I20260927 07:00:22.381037 16037 ts_tablet_manager.cc:1397] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.000s
I20260927 07:00:22.382076 16047 raft_consensus.cc:482] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:22.382242 16039 ts_tablet_manager.cc:1428] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260927 07:00:22.382258 16047 raft_consensus.cc:504] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.380162 16080 ts_tablet_manager.cc:1918] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:00:22.382704 16080 log.cc:1196] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/079d6e51cdb74be2b4102d95ebae4489
I20260927 07:00:22.383279 16080 ts_tablet_manager.cc:1939] T 079d6e51cdb74be2b4102d95ebae4489 P 247d853cb67d4b99862f8598c8c11269: Deleting consensus metadata
I20260927 07:00:22.384330 16047 leader_election.cc:288] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 247d853cb67d4b99862f8598c8c11269 (127.13.121.193:42191), 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705)
I20260927 07:00:22.385023 15517 catalog_manager.cc:4841] TS 247d853cb67d4b99862f8598c8c11269 (127.13.121.193:42191): tablet 079d6e51cdb74be2b4102d95ebae4489 (table test-table [id=bb7f454ba83c4bf894fc62fec331a669]) successfully deleted
I20260927 07:00:22.385514 16037 raft_consensus.cc:348] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.385774 16037 raft_consensus.cc:374] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:22.385854 16037 raft_consensus.cc:729] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247d853cb67d4b99862f8598c8c11269, State: Initialized, Role: FOLLOWER
I20260927 07:00:22.386077 16037 consensus_queue.cc:260] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.389428 16037 ts_tablet_manager.cc:1428] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260927 07:00:22.393182 16040 tablet_bootstrap.cc:492] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: No bootstrap required, opened a new log
I20260927 07:00:22.393353 16040 ts_tablet_manager.cc:1397] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.004s
I20260927 07:00:22.394027 16040 raft_consensus.cc:348] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.394248 16040 raft_consensus.cc:374] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:22.394302 16040 raft_consensus.cc:729] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc855808dda4646af609a92adc265ef, State: Initialized, Role: FOLLOWER
I20260927 07:00:22.394513 16040 consensus_queue.cc:260] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.396250 16040 ts_tablet_manager.cc:1428] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 07:00:22.409958 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.384967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:51755 (local address 127.13.121.195:33705)
0927 07:00:22.385152 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:22.385169 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:22.403726 (+ 18557us) server_negotiation.cc:407] Connection header received
0927 07:00:22.403960 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:22.403966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:22.404031 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:22.404107 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:22.404723 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.405771 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.408299 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.409064 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.409200 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.409275 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:22.409551 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 07:00:22.409681 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 07:00:22.410284 16085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.384719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33705 (local address 127.13.121.194:51755)
0927 07:00:22.403617 (+ 18898us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:22.403635 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:22.403819 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:22.404152 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:22.404161 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:22.404185 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:22.404552 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.404564 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.405911 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.405916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.408166 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.408176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.409259 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.409265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.409410 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.409433 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 07:00:22.409520 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 07:00:22.409698 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18763,"mutex_wait_us":511,"thread_start_us":17914,"threads_started":1}
I20260927 07:00:22.410413 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.385170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:35179 (local address 127.13.121.193:42191)
0927 07:00:22.385540 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:22.385558 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:22.403546 (+ 17988us) server_negotiation.cc:407] Connection header received
0927 07:00:22.403775 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:22.403779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:22.403836 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:22.403906 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:22.404586 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.405466 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.407856 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.409144 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.409314 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.409385 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:22.410014 (+ 629us) server_negotiation.cc:299] Negotiation successful
0927 07:00:22.410156 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260927 07:00:22.410936 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0fe38ab5746460e9b117fb1fbc42cec" candidate_uuid: "0894a2e0553f4a72ab099c180ef78c8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef" is_pre_election: true
I20260927 07:00:22.411098 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.384706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42191 (local address 127.13.121.194:35179)
0927 07:00:22.403430 (+ 18724us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:22.403455 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:22.403623 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:22.403960 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:22.403965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:22.403992 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:22.404378 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.404391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.405589 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.405595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.407681 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.407697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.409537 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.409544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.409822 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.409857 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:00:22.410586 (+ 729us) client_negotiation.cc:240] Negotiation successful
0927 07:00:22.410742 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18290,"mutex_wait_us":58,"thread_start_us":18607,"threads_started":1}
I20260927 07:00:22.411192 15968 raft_consensus.cc:2454] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0894a2e0553f4a72ab099c180ef78c8e in term 0.
I20260927 07:00:22.411676 15763 leader_election.cc:302] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0894a2e0553f4a72ab099c180ef78c8e, 5fc855808dda4646af609a92adc265ef; no voters:
I20260927 07:00:22.412284 15695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0fe38ab5746460e9b117fb1fbc42cec" candidate_uuid: "0894a2e0553f4a72ab099c180ef78c8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247d853cb67d4b99862f8598c8c11269" is_pre_election: true
I20260927 07:00:22.412643 15695 raft_consensus.cc:2454] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0894a2e0553f4a72ab099c180ef78c8e in term 0.
I20260927 07:00:22.412985 16047 raft_consensus.cc:2781] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:22.413112 16047 raft_consensus.cc:482] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:22.413231 16047 raft_consensus.cc:3037] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:22.415587 16047 raft_consensus.cc:504] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.416169 16047 leader_election.cc:288] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [CANDIDATE]: Term 1 election: Requested vote from peers 247d853cb67d4b99862f8598c8c11269 (127.13.121.193:42191), 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705)
I20260927 07:00:22.416584 15695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0fe38ab5746460e9b117fb1fbc42cec" candidate_uuid: "0894a2e0553f4a72ab099c180ef78c8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247d853cb67d4b99862f8598c8c11269"
I20260927 07:00:22.416851 15695 raft_consensus.cc:3037] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:22.416991 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0fe38ab5746460e9b117fb1fbc42cec" candidate_uuid: "0894a2e0553f4a72ab099c180ef78c8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef"
I20260927 07:00:22.417217 15968 raft_consensus.cc:3037] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:22.419092 15695 raft_consensus.cc:2454] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0894a2e0553f4a72ab099c180ef78c8e in term 1.
I20260927 07:00:22.419595 15760 leader_election.cc:302] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0894a2e0553f4a72ab099c180ef78c8e, 247d853cb67d4b99862f8598c8c11269; no voters:
I20260927 07:00:22.419874 16047 raft_consensus.cc:2781] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:22.420080 15968 raft_consensus.cc:2454] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0894a2e0553f4a72ab099c180ef78c8e in term 1.
I20260927 07:00:22.420722 16047 raft_consensus.cc:686] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 1 LEADER]: Becoming Leader. State: Replica: 0894a2e0553f4a72ab099c180ef78c8e, State: Running, Role: LEADER
I20260927 07:00:22.421088 16047 consensus_queue.cc:237] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:22.425359 15533 catalog_manager.cc:5495] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194). New cstate: current_term: 1 leader_uuid: "0894a2e0553f4a72ab099c180ef78c8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } health_report { overall_health: UNKNOWN } } }
I20260927 07:00:22.455171 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.449477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.13.121.194:41769)
0927 07:00:22.450118 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:22.450123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:22.450143 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:00:22.450225 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:22.450229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:22.450304 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:22.450451 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:22.450829 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.451650 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.452933 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.453350 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.453460 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.453892 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.454472 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.454602 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:22.454722 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 07:00:22.454807 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":310,"threads_started":1}
I20260927 07:00:22.471933 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.465658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:59495 (local address 127.13.121.193:42191)
0927 07:00:22.466155 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:22.466161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:22.466496 (+ 335us) server_negotiation.cc:407] Connection header received
0927 07:00:22.466666 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:22.466671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:22.466777 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:22.466874 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:22.467337 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.468463 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.469899 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.470597 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.470737 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.471177 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.471314 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.471393 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:22.471525 (+ 132us) server_negotiation.cc:299] Negotiation successful
0927 07:00:22.471606 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345}
I20260927 07:00:22.471989 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.465780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42191 (local address 127.13.121.194:59495)
0927 07:00:22.466396 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:22.466411 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:22.466544 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:22.466891 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:22.466894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:22.466903 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:00:22.467187 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.467198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.468625 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.468630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.469753 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.469766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.470764 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.470770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.470891 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.470969 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.471409 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.471422 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 07:00:22.471489 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 07:00:22.471642 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":388,"threads_started":1}
I20260927 07:00:22.482811 15968 raft_consensus.cc:1261] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Refusing update from remote peer 0894a2e0553f4a72ab099c180ef78c8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:22.483031 15695 raft_consensus.cc:1261] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Refusing update from remote peer 0894a2e0553f4a72ab099c180ef78c8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:22.483876 16087 consensus_queue.cc:1035] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:22.484302 16047 consensus_queue.cc:1035] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:22.521003 16075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.515545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:41769 (local address 127.13.121.193:45415)
0927 07:00:22.515763 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:22.515783 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:22.515892 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:22.516322 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:22.516327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:22.516339 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:00:22.516642 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.516659 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.517809 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.517814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.518885 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:22.518903 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.519860 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.519866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:22.520031 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.520069 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.520460 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.520471 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 07:00:22.520538 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 07:00:22.520703 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260927 07:00:22.521516 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:22.515875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:45415 (local address 127.13.121.194:41769)
0927 07:00:22.516083 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:22.516087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:22.516099 (+ 12us) server_negotiation.cc:407] Connection header received
0927 07:00:22.516146 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:22.516149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:22.516205 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:22.516278 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:22.516830 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.517679 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.519036 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:22.519715 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:22.519850 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:22.520221 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:22.520337 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:22.520412 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:22.521204 (+ 792us) server_negotiation.cc:299] Negotiation successful
0927 07:00:22.521281 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
W20260927 07:00:22.522563 16064 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 07:00:24.134176 15533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:24.134397 15533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:24.138834 15533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d: Sending DeleteTablet for 3 replicas of tablet f0fe38ab5746460e9b117fb1fbc42cec
I20260927 07:00:24.139792 15669 tablet_service.cc:1514] Processing DeleteTablet for tablet f0fe38ab5746460e9b117fb1fbc42cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:24 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:53428
I20260927 07:00:24.139792 15948 tablet_service.cc:1514] Processing DeleteTablet for tablet f0fe38ab5746460e9b117fb1fbc42cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:24 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:51534
I20260927 07:00:24.140245 15809 tablet_service.cc:1514] Processing DeleteTablet for tablet f0fe38ab5746460e9b117fb1fbc42cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:24 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35340
I20260927 07:00:24.140451 16123 tablet_replica.cc:330] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: stopping tablet replica
I20260927 07:00:24.140655 16123 raft_consensus.cc:2229] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:24.140904 16123 raft_consensus.cc:2258] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:24.140955 16125 tablet_replica.cc:330] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: stopping tablet replica
I20260927 07:00:24.141242 16125 raft_consensus.cc:2229] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:00:24.141475 16124 tablet_replica.cc:330] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: stopping tablet replica
I20260927 07:00:24.141659 16124 raft_consensus.cc:2229] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:24.141894 16124 raft_consensus.cc:2258] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:24.141958 16125 raft_consensus.cc:2258] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:24.142971 16124 ts_tablet_manager.cc:1905] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:24.142971 16125 ts_tablet_manager.cc:1905] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:24.148420 16123 ts_tablet_manager.cc:1905] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:24.149788 16125 ts_tablet_manager.cc:1918] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:00:24.149789 16124 ts_tablet_manager.cc:1918] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:00:24.149935 16125 log.cc:1196] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/f0fe38ab5746460e9b117fb1fbc42cec
I20260927 07:00:24.149946 16124 log.cc:1196] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/f0fe38ab5746460e9b117fb1fbc42cec
I20260927 07:00:24.150357 16124 ts_tablet_manager.cc:1939] T f0fe38ab5746460e9b117fb1fbc42cec P 5fc855808dda4646af609a92adc265ef: Deleting consensus metadata
I20260927 07:00:24.150357 16125 ts_tablet_manager.cc:1939] T f0fe38ab5746460e9b117fb1fbc42cec P 0894a2e0553f4a72ab099c180ef78c8e: Deleting consensus metadata
I20260927 07:00:24.151620 15518 catalog_manager.cc:4841] TS 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769): tablet f0fe38ab5746460e9b117fb1fbc42cec (table test-table [id=2c93cc2321a44c11927600d04d9d2078]) successfully deleted
I20260927 07:00:24.151623 15520 catalog_manager.cc:4841] TS 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705): tablet f0fe38ab5746460e9b117fb1fbc42cec (table test-table [id=2c93cc2321a44c11927600d04d9d2078]) successfully deleted
I20260927 07:00:24.153664 16123 ts_tablet_manager.cc:1918] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:00:24.153803 16123 log.cc:1196] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/f0fe38ab5746460e9b117fb1fbc42cec
I20260927 07:00:24.154222 16123 ts_tablet_manager.cc:1939] T f0fe38ab5746460e9b117fb1fbc42cec P 247d853cb67d4b99862f8598c8c11269: Deleting consensus metadata
I20260927 07:00:24.155380 15517 catalog_manager.cc:4841] TS 247d853cb67d4b99862f8598c8c11269 (127.13.121.193:42191): tablet f0fe38ab5746460e9b117fb1fbc42cec (table test-table [id=2c93cc2321a44c11927600d04d9d2078]) successfully deleted
Sep 27 07:00:24 dist-test-slave-p6hz krb5kdc[15488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:00:24 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:24.197826 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:24.179120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55084 (local address 127.13.121.254:39251)
0927 07:00:24.179717 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:24.179723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:24.179823 (+ 100us) server_negotiation.cc:407] Connection header received
0927 07:00:24.180015 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:24.180021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:24.180093 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:24.180183 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:24.181683 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:24.182627 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:24.184612 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:24.185378 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:24.185513 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:24.190746 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:24.190753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:24.190756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:24.190799 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:24.193973 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:24.194798 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:24.194803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:24.194805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:24.195283 (+ 478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:24.196174 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:24.196179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:24.196182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:24.196609 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:24.196823 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:24.197353 (+ 530us) server_negotiation.cc:299] Negotiation successful
0927 07:00:24.197547 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":239,"threads_started":1}
I20260927 07:00:24.201938 15533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:00:24.202533 15533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:24.210774 15669 tablet_service.cc:1467] Processing CreateTablet for tablet 00a2dc21eee244aa99df46386d0fbec5 (DEFAULT_TABLE table=test-table [id=fd0825a9c5ef4b79ad7e93097d3e0635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:24.210865 15948 tablet_service.cc:1467] Processing CreateTablet for tablet 00a2dc21eee244aa99df46386d0fbec5 (DEFAULT_TABLE table=test-table [id=fd0825a9c5ef4b79ad7e93097d3e0635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:24.210901 15809 tablet_service.cc:1467] Processing CreateTablet for tablet 00a2dc21eee244aa99df46386d0fbec5 (DEFAULT_TABLE table=test-table [id=fd0825a9c5ef4b79ad7e93097d3e0635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:24.211267 15669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00a2dc21eee244aa99df46386d0fbec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:24.211326 15948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00a2dc21eee244aa99df46386d0fbec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:24.211326 15809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00a2dc21eee244aa99df46386d0fbec5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:24.225937 16136 tablet_bootstrap.cc:492] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: Bootstrap starting.
I20260927 07:00:24.226298 16134 tablet_bootstrap.cc:492] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: Bootstrap starting.
I20260927 07:00:24.224258 16135 tablet_bootstrap.cc:492] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: Bootstrap starting.
I20260927 07:00:24.227972 16136 tablet_bootstrap.cc:654] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:24.227979 16134 tablet_bootstrap.cc:654] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:24.228288 16135 tablet_bootstrap.cc:654] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:24.240247 16136 tablet_bootstrap.cc:492] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: No bootstrap required, opened a new log
I20260927 07:00:24.240453 16136 ts_tablet_manager.cc:1397] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20260927 07:00:24.241199 16136 raft_consensus.cc:348] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.241446 16136 raft_consensus.cc:374] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:24.241559 16136 raft_consensus.cc:729] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0894a2e0553f4a72ab099c180ef78c8e, State: Initialized, Role: FOLLOWER
I20260927 07:00:24.241838 16136 consensus_queue.cc:260] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.242899 16135 tablet_bootstrap.cc:492] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: No bootstrap required, opened a new log
I20260927 07:00:24.243113 16135 ts_tablet_manager.cc:1397] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.004s
I20260927 07:00:24.243304 16134 tablet_bootstrap.cc:492] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: No bootstrap required, opened a new log
I20260927 07:00:24.243465 16134 ts_tablet_manager.cc:1397] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.010s
I20260927 07:00:24.244060 16135 raft_consensus.cc:348] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.244365 16135 raft_consensus.cc:374] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:24.244279 16134 raft_consensus.cc:348] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.244531 16134 raft_consensus.cc:374] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:24.244479 16135 raft_consensus.cc:729] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc855808dda4646af609a92adc265ef, State: Initialized, Role: FOLLOWER
I20260927 07:00:24.244632 16134 raft_consensus.cc:729] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247d853cb67d4b99862f8598c8c11269, State: Initialized, Role: FOLLOWER
I20260927 07:00:24.244946 16136 ts_tablet_manager.cc:1428] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260927 07:00:24.244910 16134 consensus_queue.cc:260] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.245492 16134 ts_tablet_manager.cc:1428] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:00:24.244901 16135 consensus_queue.cc:260] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.247789 16135 ts_tablet_manager.cc:1428] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260927 07:00:24.274210 16118 raft_consensus.cc:482] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:24.274425 16118 raft_consensus.cc:504] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.275104 16118 leader_election.cc:288] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769), 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705)
I20260927 07:00:24.275740 15834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00a2dc21eee244aa99df46386d0fbec5" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0894a2e0553f4a72ab099c180ef78c8e" is_pre_election: true
I20260927 07:00:24.275758 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00a2dc21eee244aa99df46386d0fbec5" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef" is_pre_election: true
I20260927 07:00:24.275951 15834 raft_consensus.cc:2454] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 0.
I20260927 07:00:24.276000 15968 raft_consensus.cc:2454] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 0.
I20260927 07:00:24.276436 15621 leader_election.cc:302] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0894a2e0553f4a72ab099c180ef78c8e, 247d853cb67d4b99862f8598c8c11269; no voters:
I20260927 07:00:24.276769 16118 raft_consensus.cc:2781] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:24.276897 16118 raft_consensus.cc:482] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:24.276991 16118 raft_consensus.cc:3037] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:24.279086 16118 raft_consensus.cc:504] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.279654 16118 leader_election.cc:288] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 election: Requested vote from peers 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769), 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705)
I20260927 07:00:24.280104 15834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00a2dc21eee244aa99df46386d0fbec5" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0894a2e0553f4a72ab099c180ef78c8e"
I20260927 07:00:24.280153 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00a2dc21eee244aa99df46386d0fbec5" candidate_uuid: "247d853cb67d4b99862f8598c8c11269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef"
I20260927 07:00:24.280289 15834 raft_consensus.cc:3037] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:24.280313 15968 raft_consensus.cc:3037] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:24.282446 15968 raft_consensus.cc:2454] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 1.
I20260927 07:00:24.282583 15834 raft_consensus.cc:2454] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 247d853cb67d4b99862f8598c8c11269 in term 1.
I20260927 07:00:24.282912 15623 leader_election.cc:302] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 247d853cb67d4b99862f8598c8c11269, 5fc855808dda4646af609a92adc265ef; no voters:
I20260927 07:00:24.283316 16118 raft_consensus.cc:2781] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:24.283447 16118 raft_consensus.cc:686] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 1 LEADER]: Becoming Leader. State: Replica: 247d853cb67d4b99862f8598c8c11269, State: Running, Role: LEADER
I20260927 07:00:24.283684 16118 consensus_queue.cc:237] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.286268 15533 catalog_manager.cc:5495] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 reported cstate change: term changed from 0 to 1, leader changed from <none> to 247d853cb67d4b99862f8598c8c11269 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "247d853cb67d4b99862f8598c8c11269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } health_report { overall_health: UNKNOWN } } }
I20260927 07:00:24.316299 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:24.310944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.13.121.193:42191)
0927 07:00:24.311624 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:24.311628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:24.311647 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:00:24.311700 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:24.311704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:24.311772 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:24.311858 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:24.312225 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:24.312978 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:24.314342 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:24.314783 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:24.314881 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:24.315401 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:24.315536 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:24.315614 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:24.315847 (+ 233us) server_negotiation.cc:299] Negotiation successful
0927 07:00:24.315919 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":419,"threads_started":1}
I20260927 07:00:24.325048 15968 raft_consensus.cc:1261] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Refusing update from remote peer 247d853cb67d4b99862f8598c8c11269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:24.325153 15834 raft_consensus.cc:1261] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Refusing update from remote peer 247d853cb67d4b99862f8598c8c11269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:24.325697 16145 consensus_queue.cc:1035] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:24.325971 16102 consensus_queue.cc:1035] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:24.368721 15532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:24.368943 15532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:24.371858 15532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d: Sending DeleteTablet for 3 replicas of tablet 00a2dc21eee244aa99df46386d0fbec5
I20260927 07:00:24.372696 15671 tablet_service.cc:1514] Processing DeleteTablet for tablet 00a2dc21eee244aa99df46386d0fbec5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:24 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:53428
I20260927 07:00:24.372785 15809 tablet_service.cc:1514] Processing DeleteTablet for tablet 00a2dc21eee244aa99df46386d0fbec5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:24 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35340
I20260927 07:00:24.372785 15948 tablet_service.cc:1514] Processing DeleteTablet for tablet 00a2dc21eee244aa99df46386d0fbec5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:24 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:51534
I20260927 07:00:24.372974 16123 tablet_replica.cc:330] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: stopping tablet replica
I20260927 07:00:24.373054 16124 tablet_replica.cc:330] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: stopping tablet replica
I20260927 07:00:24.373054 16125 tablet_replica.cc:330] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: stopping tablet replica
I20260927 07:00:24.373210 16123 raft_consensus.cc:2229] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:00:24.373237 16124 raft_consensus.cc:2229] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:24.373234 16125 raft_consensus.cc:2229] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:24.373416 16125 raft_consensus.cc:2258] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:24.373425 16124 raft_consensus.cc:2258] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:24.373523 16123 raft_consensus.cc:2258] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:24.374645 16123 ts_tablet_manager.cc:1905] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:24.374645 16125 ts_tablet_manager.cc:1905] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:24.375334 15532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:00:24.376044 15532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:24.377048 16124 ts_tablet_manager.cc:1905] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:24.381086 16123 ts_tablet_manager.cc:1918] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:00:24.381245 16123 log.cc:1196] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/00a2dc21eee244aa99df46386d0fbec5
I20260927 07:00:24.381659 16123 ts_tablet_manager.cc:1939] T 00a2dc21eee244aa99df46386d0fbec5 P 247d853cb67d4b99862f8598c8c11269: Deleting consensus metadata
I20260927 07:00:24.382587 15671 tablet_service.cc:1467] Processing CreateTablet for tablet 77931374c2d34ab49465c563e1317ca5 (DEFAULT_TABLE table=test-table [id=51a69e869aa64135afaaa79d246b5d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:24.382727 15517 catalog_manager.cc:4841] TS 247d853cb67d4b99862f8598c8c11269 (127.13.121.193:42191): tablet 00a2dc21eee244aa99df46386d0fbec5 (table test-table [id=fd0825a9c5ef4b79ad7e93097d3e0635]) successfully deleted
I20260927 07:00:24.383001 15671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77931374c2d34ab49465c563e1317ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:24.383927 15809 tablet_service.cc:1467] Processing CreateTablet for tablet 77931374c2d34ab49465c563e1317ca5 (DEFAULT_TABLE table=test-table [id=51a69e869aa64135afaaa79d246b5d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:24.384320 15809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77931374c2d34ab49465c563e1317ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:24.387283 16125 ts_tablet_manager.cc:1918] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:00:24.387315 16134 tablet_bootstrap.cc:492] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: Bootstrap starting.
I20260927 07:00:24.387446 16125 log.cc:1196] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/00a2dc21eee244aa99df46386d0fbec5
I20260927 07:00:24.388670 16134 tablet_bootstrap.cc:654] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:24.389444 16124 ts_tablet_manager.cc:1918] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:00:24.389585 16124 log.cc:1196] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/00a2dc21eee244aa99df46386d0fbec5
I20260927 07:00:24.389955 16124 ts_tablet_manager.cc:1939] T 00a2dc21eee244aa99df46386d0fbec5 P 5fc855808dda4646af609a92adc265ef: Deleting consensus metadata
I20260927 07:00:24.390581 16136 tablet_bootstrap.cc:492] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: Bootstrap starting.
I20260927 07:00:24.390637 15948 tablet_service.cc:1467] Processing CreateTablet for tablet 77931374c2d34ab49465c563e1317ca5 (DEFAULT_TABLE table=test-table [id=51a69e869aa64135afaaa79d246b5d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:24.390925 16134 tablet_bootstrap.cc:492] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: No bootstrap required, opened a new log
I20260927 07:00:24.391047 15948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77931374c2d34ab49465c563e1317ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:24.391054 16125 ts_tablet_manager.cc:1939] T 00a2dc21eee244aa99df46386d0fbec5 P 0894a2e0553f4a72ab099c180ef78c8e: Deleting consensus metadata
I20260927 07:00:24.391086 16134 ts_tablet_manager.cc:1397] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260927 07:00:24.391844 16134 raft_consensus.cc:348] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.392091 16134 raft_consensus.cc:374] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:24.392162 16134 raft_consensus.cc:729] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247d853cb67d4b99862f8598c8c11269, State: Initialized, Role: FOLLOWER
I20260927 07:00:24.392225 16136 tablet_bootstrap.cc:654] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:24.392179 15518 catalog_manager.cc:4841] TS 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769): tablet 00a2dc21eee244aa99df46386d0fbec5 (table test-table [id=fd0825a9c5ef4b79ad7e93097d3e0635]) successfully deleted
I20260927 07:00:24.392371 16134 consensus_queue.cc:260] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.392864 16134 ts_tablet_manager.cc:1428] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:00:24.393690 15520 catalog_manager.cc:4841] TS 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705): tablet 00a2dc21eee244aa99df46386d0fbec5 (table test-table [id=fd0825a9c5ef4b79ad7e93097d3e0635]) successfully deleted
I20260927 07:00:24.394193 16136 tablet_bootstrap.cc:492] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: No bootstrap required, opened a new log
I20260927 07:00:24.394315 16136 ts_tablet_manager.cc:1397] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260927 07:00:24.394972 16136 raft_consensus.cc:348] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.395247 16136 raft_consensus.cc:374] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:24.395329 16136 raft_consensus.cc:729] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0894a2e0553f4a72ab099c180ef78c8e, State: Initialized, Role: FOLLOWER
I20260927 07:00:24.395589 16136 consensus_queue.cc:260] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.396312 16136 ts_tablet_manager.cc:1428] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:00:24.397451 16135 tablet_bootstrap.cc:492] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: Bootstrap starting.
I20260927 07:00:24.398885 16135 tablet_bootstrap.cc:654] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:24.400763 16135 tablet_bootstrap.cc:492] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: No bootstrap required, opened a new log
I20260927 07:00:24.400915 16135 ts_tablet_manager.cc:1397] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260927 07:00:24.401736 16135 raft_consensus.cc:348] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.401979 16135 raft_consensus.cc:374] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:24.402053 16135 raft_consensus.cc:729] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc855808dda4646af609a92adc265ef, State: Initialized, Role: FOLLOWER
I20260927 07:00:24.402262 16135 consensus_queue.cc:260] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.402693 16135 ts_tablet_manager.cc:1428] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260927 07:00:24.479591 16117 raft_consensus.cc:482] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:24.479802 16117 raft_consensus.cc:504] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.480422 16117 leader_election.cc:288] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 247d853cb67d4b99862f8598c8c11269 (127.13.121.193:42191), 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705)
I20260927 07:00:24.480957 15694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77931374c2d34ab49465c563e1317ca5" candidate_uuid: "0894a2e0553f4a72ab099c180ef78c8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247d853cb67d4b99862f8598c8c11269" is_pre_election: true
I20260927 07:00:24.481006 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77931374c2d34ab49465c563e1317ca5" candidate_uuid: "0894a2e0553f4a72ab099c180ef78c8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef" is_pre_election: true
I20260927 07:00:24.481233 15694 raft_consensus.cc:2454] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0894a2e0553f4a72ab099c180ef78c8e in term 0.
I20260927 07:00:24.481257 15968 raft_consensus.cc:2454] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0894a2e0553f4a72ab099c180ef78c8e in term 0.
I20260927 07:00:24.481632 15763 leader_election.cc:302] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0894a2e0553f4a72ab099c180ef78c8e, 5fc855808dda4646af609a92adc265ef; no voters:
I20260927 07:00:24.481890 16117 raft_consensus.cc:2781] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:24.482009 16117 raft_consensus.cc:482] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:24.482105 16117 raft_consensus.cc:3037] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:24.484387 16117 raft_consensus.cc:504] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.484942 16117 leader_election.cc:288] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [CANDIDATE]: Term 1 election: Requested vote from peers 247d853cb67d4b99862f8598c8c11269 (127.13.121.193:42191), 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705)
I20260927 07:00:24.485488 15968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77931374c2d34ab49465c563e1317ca5" candidate_uuid: "0894a2e0553f4a72ab099c180ef78c8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc855808dda4646af609a92adc265ef"
I20260927 07:00:24.485487 15694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77931374c2d34ab49465c563e1317ca5" candidate_uuid: "0894a2e0553f4a72ab099c180ef78c8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247d853cb67d4b99862f8598c8c11269"
I20260927 07:00:24.485679 15968 raft_consensus.cc:3037] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:24.485700 15694 raft_consensus.cc:3037] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:24.487737 15694 raft_consensus.cc:2454] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0894a2e0553f4a72ab099c180ef78c8e in term 1.
I20260927 07:00:24.487735 15968 raft_consensus.cc:2454] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0894a2e0553f4a72ab099c180ef78c8e in term 1.
I20260927 07:00:24.488180 15760 leader_election.cc:302] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0894a2e0553f4a72ab099c180ef78c8e, 247d853cb67d4b99862f8598c8c11269; no voters:
I20260927 07:00:24.488478 16117 raft_consensus.cc:2781] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:24.488615 16117 raft_consensus.cc:686] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 LEADER]: Becoming Leader. State: Replica: 0894a2e0553f4a72ab099c180ef78c8e, State: Running, Role: LEADER
I20260927 07:00:24.488898 16117 consensus_queue.cc:237] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } }
I20260927 07:00:24.491458 15533 catalog_manager.cc:5495] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194). New cstate: current_term: 1 leader_uuid: "0894a2e0553f4a72ab099c180ef78c8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0894a2e0553f4a72ab099c180ef78c8e" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 41769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 } health_report { overall_health: UNKNOWN } } }
I20260927 07:00:24.517592 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:24.512226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.13.121.194:41769)
0927 07:00:24.512779 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:24.512783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:24.512801 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:00:24.512861 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:24.512865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:24.512924 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:24.513050 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:24.513453 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:24.514347 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:24.515813 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:24.516258 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:24.516361 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:24.516867 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:24.516989 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:24.517058 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:24.517206 (+ 148us) server_negotiation.cc:299] Negotiation successful
0927 07:00:24.517274 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":299,"threads_started":1}
I20260927 07:00:24.526463 15968 raft_consensus.cc:1261] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Refusing update from remote peer 0894a2e0553f4a72ab099c180ef78c8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:24.526633 15694 raft_consensus.cc:1261] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Refusing update from remote peer 0894a2e0553f4a72ab099c180ef78c8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:24.527055 16117 consensus_queue.cc:1035] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc855808dda4646af609a92adc265ef" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:24.527361 16111 consensus_queue.cc:1035] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "247d853cb67d4b99862f8598c8c11269" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 07:00:24.546902 16104 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 07:00:26.132247 15533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:26.132501 15533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:26.137209 15533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1eb5344764e4bf49590eb212d9bd28d: Sending DeleteTablet for 3 replicas of tablet 77931374c2d34ab49465c563e1317ca5
I20260927 07:00:26.138100 15809 tablet_service.cc:1514] Processing DeleteTablet for tablet 77931374c2d34ab49465c563e1317ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:26 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35340
I20260927 07:00:26.138106 15671 tablet_service.cc:1514] Processing DeleteTablet for tablet 77931374c2d34ab49465c563e1317ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:26 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:53428
I20260927 07:00:26.138263 15948 tablet_service.cc:1514] Processing DeleteTablet for tablet 77931374c2d34ab49465c563e1317ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:26 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:51534
I20260927 07:00:26.138875 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15604
I20260927 07:00:26.151820 16178 tablet_replica.cc:330] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: stopping tablet replica
I20260927 07:00:26.152207 16178 raft_consensus.cc:2229] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:26.152565 16178 raft_consensus.cc:2258] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:26.153168 16179 tablet_replica.cc:330] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: stopping tablet replica
I20260927 07:00:26.153635 16177 tablet_replica.cc:330] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: stopping tablet replica
I20260927 07:00:26.153832 16177 raft_consensus.cc:2229] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:00:26.153867 16178 ts_tablet_manager.cc:1905] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:26.154209 16177 raft_consensus.cc:2258] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:26.155119 16177 ts_tablet_manager.cc:1905] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:26.158013 16179 raft_consensus.cc:2229] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:26.158354 16179 raft_consensus.cc:2258] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 27 07:00:26 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:26.159535 16179 ts_tablet_manager.cc:1905] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:26.160584 16177 ts_tablet_manager.cc:1918] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:00:26.160646 16178 ts_tablet_manager.cc:1918] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:00:26.160728 16177 log.cc:1196] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/77931374c2d34ab49465c563e1317ca5
I20260927 07:00:26.160773 16178 log.cc:1196] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/77931374c2d34ab49465c563e1317ca5
I20260927 07:00:26.161180 16177 ts_tablet_manager.cc:1939] T 77931374c2d34ab49465c563e1317ca5 P 0894a2e0553f4a72ab099c180ef78c8e: Deleting consensus metadata
I20260927 07:00:26.161201 16178 ts_tablet_manager.cc:1939] T 77931374c2d34ab49465c563e1317ca5 P 247d853cb67d4b99862f8598c8c11269: Deleting consensus metadata
I20260927 07:00:26.162310 15517 catalog_manager.cc:4841] TS 247d853cb67d4b99862f8598c8c11269 (127.13.121.193:42191): tablet 77931374c2d34ab49465c563e1317ca5 (table test-table [id=51a69e869aa64135afaaa79d246b5d02]) successfully deleted
I20260927 07:00:26.162313 15518 catalog_manager.cc:4841] TS 0894a2e0553f4a72ab099c180ef78c8e (127.13.121.194:41769): tablet 77931374c2d34ab49465c563e1317ca5 (table test-table [id=51a69e869aa64135afaaa79d246b5d02]) successfully deleted
I20260927 07:00:26.165459 16179 ts_tablet_manager.cc:1918] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 07:00:26.165601 16179 log.cc:1196] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/77931374c2d34ab49465c563e1317ca5
I20260927 07:00:26.166070 16179 ts_tablet_manager.cc:1939] T 77931374c2d34ab49465c563e1317ca5 P 5fc855808dda4646af609a92adc265ef: Deleting consensus metadata
I20260927 07:00:26.166481 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:26.140735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.13.121.193:42191)
0927 07:00:26.148212 (+ 7477us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:26.148218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:26.148243 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:00:26.148308 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:26.148314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:26.148395 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:26.148531 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:26.150433 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:26.151692 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:26.153408 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:26.154030 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:26.154193 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:26.159801 (+ 5608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:26.159823 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:26.159845 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:26.159897 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:26.163596 (+ 3699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:26.164283 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:26.164291 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:26.164293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:26.164422 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:26.164822 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:26.164825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:26.164827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:26.165215 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:26.165471 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:26.165849 (+ 378us) server_negotiation.cc:299] Negotiation successful
0927 07:00:26.166102 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7326,"thread_start_us":7174,"threads_started":1}
W20260927 07:00:26.167078 15735 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53474
I20260927 07:00:26.167083 15520 catalog_manager.cc:4841] TS 5fc855808dda4646af609a92adc265ef (127.13.121.195:33705): tablet 77931374c2d34ab49465c563e1317ca5 (table test-table [id=51a69e869aa64135afaaa79d246b5d02]) successfully deleted
W20260927 07:00:26.167670 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15604: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:00:26.167865 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 15604
I20260927 07:00:26.188917 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15744
Sep 27 07:00:26 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:00:26.203948 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:26.189682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.13.121.194:41769)
0927 07:00:26.190137 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:26.190141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:26.190157 (+ 16us) server_negotiation.cc:407] Connection header received
0927 07:00:26.190210 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:26.190214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:26.190270 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:26.190358 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:26.191668 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:26.192395 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:26.193756 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:26.194192 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:26.194288 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:26.198300 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:26.198316 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:26.198332 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:26.198371 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:26.201129 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:26.201937 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:26.201945 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:26.201947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:26.202086 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:26.202496 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:26.202500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:26.202502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:26.202886 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:26.203129 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:26.203506 (+ 377us) server_negotiation.cc:299] Negotiation successful
0927 07:00:26.203687 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":239,"threads_started":1}
W20260927 07:00:26.204594 15874 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35366
W20260927 07:00:26.205189 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15744: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:00:26.205320 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 15744
I20260927 07:00:26.224001 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15883
Sep 27 07:00:26 dist-test-slave-p6hz krb5kdc[15488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:00:26.239615 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:26.224737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51550 (local address 127.13.121.195:33705)
0927 07:00:26.225269 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:26.225273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:26.225287 (+ 14us) server_negotiation.cc:407] Connection header received
0927 07:00:26.225338 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:26.225342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:26.225399 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:26.225492 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:26.226836 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:26.227663 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:26.228930 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:26.229393 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:26.229591 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:26.234003 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:26.234019 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:26.234035 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:26.234082 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:26.236825 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:26.237583 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:26.237591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:26.237593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:26.237732 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:26.238086 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:26.238091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:26.238094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:26.238455 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:26.238735 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:26.239016 (+ 281us) server_negotiation.cc:299] Negotiation successful
0927 07:00:26.239286 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":238,"threads_started":1}
W20260927 07:00:26.240222 16013 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51550
W20260927 07:00:26.240829 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15883: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:00:26.240983 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 15883
I20260927 07:00:26.255057 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 15501
I20260927 07:00:26.256383 15573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:26.370059 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 15501
2026-09-27T07:00:26Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7851 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:26 dist-test-slave-p6hz krb5kdc[16187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:26 dist-test-slave-p6hz krb5kdc[16187](info): set up 2 sockets
Sep 27 07:00:26 dist-test-slave-p6hz krb5kdc[16187](info): commencing operation
krb5kdc: starting...
W20260927 07:00:27.379405 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:27 dist-test-slave-p6hz krb5kdc[16187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:27.567288 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38093
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:38093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:27.896617 16200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:27.897017 16200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:27.897142 16200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:27.904644 16200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:27.904781 16200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:27.904860 16200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:27.904939 16200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:27.904999 16200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:27.915336 16200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:38093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:27.916728 16200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:27.918102 16200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:27.930462 16205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:27.930488 16208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:27.930531 16206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:27.931318 16200 server_base.cc:1034] running on GCE node
I20260927 07:00:27.932175 16200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:27.933765 16200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:27.935011 16200 hybrid_clock.cc:648] HybridClock initialized: now 1790492427934940 us; error 59 us; skew 500 ppm
I20260927 07:00:27.935451 16200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:27 dist-test-slave-p6hz krb5kdc[16187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492427, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:27.943033 16200 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:27.944751 16200 webserver.cc:460] Webserver started at http://127.13.121.254:43571/ using document root <none> and password file <none>
I20260927 07:00:27.945508 16200 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:27.945641 16200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:27.945960 16200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:27.949254 16200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "20e3c223fdb44db6b38796332847776b"
format_stamp: "Formatted at 2026-09-27 07:00:27 on dist-test-slave-p6hz"
server_key: "72508d5dda9e7dcc8e1d35e5ceb31146"
server_key_iv: "96f8d9af57c19791459d3ddd242bd24f"
server_key_version: "encryptionkey@0"
I20260927 07:00:27.950170 16200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "20e3c223fdb44db6b38796332847776b"
format_stamp: "Formatted at 2026-09-27 07:00:27 on dist-test-slave-p6hz"
server_key: "72508d5dda9e7dcc8e1d35e5ceb31146"
server_key_iv: "96f8d9af57c19791459d3ddd242bd24f"
server_key_version: "encryptionkey@0"
I20260927 07:00:27.955979 16200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:00:27.959712 16215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:27.961200 16200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260927 07:00:27.961385 16200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "20e3c223fdb44db6b38796332847776b"
format_stamp: "Formatted at 2026-09-27 07:00:27 on dist-test-slave-p6hz"
server_key: "72508d5dda9e7dcc8e1d35e5ceb31146"
server_key_iv: "96f8d9af57c19791459d3ddd242bd24f"
server_key_version: "encryptionkey@0"
I20260927 07:00:27.961580 16200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:27.985263 16200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:27.989656 16200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:27.989926 16200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:28.007215 16200 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:38093
I20260927 07:00:28.007241 16266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:38093 every 8 connection(s)
I20260927 07:00:28.008622 16200 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:28.012593 16267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:28.015491 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 16200
I20260927 07:00:28.015704 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:28.016340 13799 external_mini_cluster.cc:1428] Setting key 587aa777f0b457e6a4371fcfe4993b6c
I20260927 07:00:28.025893 16267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b: Bootstrap starting.
I20260927 07:00:28.029644 16267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:28.030958 16267 log.cc:824] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b: Log is configured to *not* fsync() on all Append() calls
Sep 27 07:00:28 dist-test-slave-p6hz krb5kdc[16187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:28.034240 16267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b: No bootstrap required, opened a new log
I20260927 07:00:28.043046 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:28.018572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58798 (local address 127.13.121.254:38093)
0927 07:00:28.019409 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:28.019426 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:28.019479 (+ 53us) server_negotiation.cc:407] Connection header received
0927 07:00:28.021543 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:28.021576 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:28.022181 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:28.022676 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:28.024130 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:28.026095 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:28.027484 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:28.028590 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:28.028788 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:28.033424 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:28.033434 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:28.033451 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:28.033489 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:28.037458 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:28.038402 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:28.038411 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:28.038422 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:28.038658 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:28.039102 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:28.039107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:28.039110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:28.039924 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:28.040222 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:28.040720 (+ 498us) server_negotiation.cc:299] Negotiation successful
0927 07:00:28.041477 (+ 757us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":318,"threads_started":1}
I20260927 07:00:28.044620 16267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e3c223fdb44db6b38796332847776b" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38093 } }
I20260927 07:00:28.044997 16267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:28.045097 16267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e3c223fdb44db6b38796332847776b, State: Initialized, Role: FOLLOWER
I20260927 07:00:28.045683 16267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e3c223fdb44db6b38796332847776b" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38093 } }
I20260927 07:00:28.045936 16267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:28.046120 16267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:28.046295 16267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:28.048630 16267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e3c223fdb44db6b38796332847776b" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38093 } }
I20260927 07:00:28.049206 16267 leader_election.cc:302] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20e3c223fdb44db6b38796332847776b; no voters:
I20260927 07:00:28.049804 16267 leader_election.cc:288] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:28.049889 16272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:28.050549 16272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [term 1 LEADER]: Becoming Leader. State: Replica: 20e3c223fdb44db6b38796332847776b, State: Running, Role: LEADER
I20260927 07:00:28.051079 16272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e3c223fdb44db6b38796332847776b" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38093 } }
I20260927 07:00:28.051848 16267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:28.053740 16274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20e3c223fdb44db6b38796332847776b. Latest consensus state: current_term: 1 leader_uuid: "20e3c223fdb44db6b38796332847776b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e3c223fdb44db6b38796332847776b" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38093 } } }
I20260927 07:00:28.054148 16274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:28.053758 16273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20e3c223fdb44db6b38796332847776b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e3c223fdb44db6b38796332847776b" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38093 } } }
I20260927 07:00:28.055557 16273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:28.057847 16278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:28.064091 16278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:28.073983 16278 catalog_manager.cc:1347] Generated new cluster ID: 981d06c632d648a6b9a3246c1e1d4588
I20260927 07:00:28.074112 16278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:28.082967 16278 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:28.084988 16278 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:28.098111 16278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20e3c223fdb44db6b38796332847776b: Generated new TSK 0
I20260927 07:00:28.099432 16278 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:00:28.151875 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38093
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:28.491099 16293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:28.491405 16293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:28.491496 16293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:28.499416 16293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:28.499540 16293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:28.499663 16293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:28.509528 16293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:28.510825 16293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:28.512212 16293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:28.525674 16299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:28.526012 16301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:28.526398 16298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:28.526908 16293 server_base.cc:1034] running on GCE node
I20260927 07:00:28.527590 16293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:28.528649 16293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:28.529899 16293 hybrid_clock.cc:648] HybridClock initialized: now 1790492428529812 us; error 82 us; skew 500 ppm
I20260927 07:00:28.530233 16293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:28 dist-test-slave-p6hz krb5kdc[16187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492428, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:28.533890 16293 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:00:28.535506 16293 webserver.cc:460] Webserver started at http://127.13.121.193:42421/ using document root <none> and password file <none>
I20260927 07:00:28.536234 16293 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:28.536355 16293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:28.536667 16293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:28.539908 16293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "1d667ce4dab845c4bb618896a08f979a"
format_stamp: "Formatted at 2026-09-27 07:00:28 on dist-test-slave-p6hz"
server_key: "a15187e1735c74aaf0f9f1b0baa4cee7"
server_key_iv: "c960916c5214777c9190a2fc6fe022c5"
server_key_version: "encryptionkey@0"
I20260927 07:00:28.540819 16293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "1d667ce4dab845c4bb618896a08f979a"
format_stamp: "Formatted at 2026-09-27 07:00:28 on dist-test-slave-p6hz"
server_key: "a15187e1735c74aaf0f9f1b0baa4cee7"
server_key_iv: "c960916c5214777c9190a2fc6fe022c5"
server_key_version: "encryptionkey@0"
I20260927 07:00:28.546310 16293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260927 07:00:28.550099 16308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:28.551666 16293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:00:28.551838 16293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "1d667ce4dab845c4bb618896a08f979a"
format_stamp: "Formatted at 2026-09-27 07:00:28 on dist-test-slave-p6hz"
server_key: "a15187e1735c74aaf0f9f1b0baa4cee7"
server_key_iv: "c960916c5214777c9190a2fc6fe022c5"
server_key_version: "encryptionkey@0"
I20260927 07:00:28.552103 16293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:28.569056 16293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:28.573544 16293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:28.573824 16293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:28.574847 16293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:28.576294 16293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:28.576376 16293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:28.576503 16293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:28.576555 16293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:28.606400 16293 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:41557
I20260927 07:00:28.606436 16420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:41557 every 8 connection(s)
I20260927 07:00:28.607677 16293 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:28.611421 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 16293
I20260927 07:00:28.611636 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:28.612275 13799 external_mini_cluster.cc:1428] Setting key 8b7badcb59765e80dad3db9a908ee4cd
Sep 27 07:00:28 dist-test-slave-p6hz krb5kdc[16187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492428, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:00:28.628330 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:28.610247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:45523 (local address 127.13.121.254:38093)
0927 07:00:28.610687 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:28.610692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:28.611793 (+ 1101us) server_negotiation.cc:407] Connection header received
0927 07:00:28.613640 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:28.613646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:28.613732 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:28.613822 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:28.615485 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:28.616228 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:28.617873 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:28.618350 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:28.618476 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:28.622246 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:28.622252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:28.622255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:28.622294 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:28.624948 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:28.625817 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:28.625822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:28.625825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:28.625950 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:28.626443 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:28.626447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:28.626450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:28.626747 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:28.626882 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:28.627715 (+ 833us) server_negotiation.cc:299] Negotiation successful
0927 07:00:28.627960 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":222,"threads_started":1}
I20260927 07:00:28.629364 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:28.610755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38093 (local address 127.13.121.193:45523)
0927 07:00:28.611620 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:28.611668 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:28.613365 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:28.614087 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:28.614095 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:28.614618 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:28.615273 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:28.615295 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:28.616416 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:28.616421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:28.617717 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:28.617730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:28.618598 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:28.618605 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:28.618848 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:28.619531 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:28.619558 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:28.621978 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:28.625149 (+ 3171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:28.625161 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:28.625179 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:28.625676 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:28.626071 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:28.626076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:28.626081 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:28.626287 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:28.626879 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:28.626888 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:28.627304 (+ 416us) client_negotiation.cc:769] Sending connection context
0927 07:00:28.627563 (+ 259us) client_negotiation.cc:240] Negotiation successful
0927 07:00:28.628152 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":370,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
I20260927 07:00:28.631076 16421 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38093
I20260927 07:00:28.631451 16421 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:28.632217 16421 heartbeater.cc:511] Master 127.13.121.254:38093 requested a full tablet report, sending...
I20260927 07:00:28.634573 16232 ts_manager.cc:194] Registered new tserver with Master: 1d667ce4dab845c4bb618896a08f979a (127.13.121.193:41557)
I20260927 07:00:28.636137 16232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:45523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:00:28.654686 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38093
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:28.978799 16427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:28.979118 16427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:28.979271 16427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:28.986713 16427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:28.986843 16427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:28.986994 16427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:28.999891 16427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:29.001295 16427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:29.002580 16427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:29.015916 16432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:29.016109 16435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:29.015939 16433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:29.017283 16427 server_base.cc:1034] running on GCE node
I20260927 07:00:29.017923 16427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:29.018957 16427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:29.020180 16427 hybrid_clock.cc:648] HybridClock initialized: now 1790492429020122 us; error 52 us; skew 500 ppm
I20260927 07:00:29.020507 16427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:29 dist-test-slave-p6hz krb5kdc[16187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492429, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:29.023540 16427 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:00:29.024971 16427 webserver.cc:460] Webserver started at http://127.13.121.194:33295/ using document root <none> and password file <none>
I20260927 07:00:29.025614 16427 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:29.025708 16427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:29.025960 16427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:29.029050 16427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "af049d957ff6470cb182d19d7643ec40"
format_stamp: "Formatted at 2026-09-27 07:00:29 on dist-test-slave-p6hz"
server_key: "a2c0913b9a6be6037617b8f92a4e3a5a"
server_key_iv: "7a5d8cc832bd4219757e898f0b182e50"
server_key_version: "encryptionkey@0"
I20260927 07:00:29.029825 16427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "af049d957ff6470cb182d19d7643ec40"
format_stamp: "Formatted at 2026-09-27 07:00:29 on dist-test-slave-p6hz"
server_key: "a2c0913b9a6be6037617b8f92a4e3a5a"
server_key_iv: "7a5d8cc832bd4219757e898f0b182e50"
server_key_version: "encryptionkey@0"
I20260927 07:00:29.035202 16427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:00:29.038683 16442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:29.040174 16427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 07:00:29.040333 16427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "af049d957ff6470cb182d19d7643ec40"
format_stamp: "Formatted at 2026-09-27 07:00:29 on dist-test-slave-p6hz"
server_key: "a2c0913b9a6be6037617b8f92a4e3a5a"
server_key_iv: "7a5d8cc832bd4219757e898f0b182e50"
server_key_version: "encryptionkey@0"
I20260927 07:00:29.040544 16427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:29.056974 16427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:29.061513 16427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:29.061831 16427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:29.062827 16427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:29.064275 16427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:29.064348 16427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:29.064435 16427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:29.064481 16427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:29.094451 16427 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:37251
I20260927 07:00:29.094470 16554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:37251 every 8 connection(s)
I20260927 07:00:29.095971 16427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:29.102850 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 16427
I20260927 07:00:29.103060 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:29.103636 13799 external_mini_cluster.cc:1428] Setting key 88eabb11b041cc295c3d92d300641070
Sep 27 07:00:29 dist-test-slave-p6hz krb5kdc[16187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492429, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:29.117522 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:29.098724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:37799 (local address 127.13.121.254:38093)
0927 07:00:29.098890 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:29.098895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:29.100234 (+ 1339us) server_negotiation.cc:407] Connection header received
0927 07:00:29.102064 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:29.102069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:29.102155 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:29.102243 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:29.104315 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:29.105185 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:29.106821 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:29.107500 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:29.107625 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:29.110957 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:29.110964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:29.110968 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:29.111009 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:29.114136 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:29.114982 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:29.114987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:29.114990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:29.115160 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:29.115650 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:29.115656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:29.115659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:29.116001 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:29.116135 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:29.116917 (+ 782us) server_negotiation.cc:299] Negotiation successful
0927 07:00:29.117167 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:00:29.118464 16557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:29.099287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38093 (local address 127.13.121.194:37799)
0927 07:00:29.100065 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:29.100105 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:29.101768 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:29.102515 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:29.102527 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:29.103308 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:29.104131 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:29.104157 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:29.105338 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:29.105343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:29.106670 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:29.106683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:29.107694 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:29.107699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:29.107916 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:29.108552 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:29.108575 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:29.110715 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:29.114348 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:29.114359 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:29.114377 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:29.114824 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:29.115314 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:29.115318 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:29.115321 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:29.115533 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:29.116167 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:29.116174 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:29.116552 (+ 378us) client_negotiation.cc:769] Sending connection context
0927 07:00:29.116823 (+ 271us) client_negotiation.cc:240] Negotiation successful
0927 07:00:29.117309 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":303,"threads_started":1}
I20260927 07:00:29.120044 16555 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38093
I20260927 07:00:29.120348 16555 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:00:29.121085 16555 heartbeater.cc:511] Master 127.13.121.254:38093 requested a full tablet report, sending...
I20260927 07:00:29.122601 16232 ts_manager.cc:194] Registered new tserver with Master: af049d957ff6470cb182d19d7643ec40 (127.13.121.194:37251)
I20260927 07:00:29.123549 16232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:37799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:00:29.147471 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38093
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:29.469252 16560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:29.469573 16560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:29.469695 16560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:29.477129 16560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:29.477252 16560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:29.477393 16560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:29.487314 16560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:29.488710 16560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:29.490026 16560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:29.503895 16565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:29.503989 16568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:29.503994 16566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:29.505097 16560 server_base.cc:1034] running on GCE node
I20260927 07:00:29.505744 16560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:29.506807 16560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:29.508033 16560 hybrid_clock.cc:648] HybridClock initialized: now 1790492429507981 us; error 46 us; skew 500 ppm
I20260927 07:00:29.508355 16560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:29 dist-test-slave-p6hz krb5kdc[16187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492429, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:29.511541 16560 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:00:29.513120 16560 webserver.cc:460] Webserver started at http://127.13.121.195:41253/ using document root <none> and password file <none>
I20260927 07:00:29.513856 16560 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:29.513983 16560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:29.514281 16560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:29.517302 16560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "1da2d1e2148d4e4eae1fb1ba89543082"
format_stamp: "Formatted at 2026-09-27 07:00:29 on dist-test-slave-p6hz"
server_key: "bffb37ac49e8b502f3af22d3812ce592"
server_key_iv: "5b72c3e85adb0d536b4916165c28782f"
server_key_version: "encryptionkey@0"
I20260927 07:00:29.518121 16560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "1da2d1e2148d4e4eae1fb1ba89543082"
format_stamp: "Formatted at 2026-09-27 07:00:29 on dist-test-slave-p6hz"
server_key: "bffb37ac49e8b502f3af22d3812ce592"
server_key_iv: "5b72c3e85adb0d536b4916165c28782f"
server_key_version: "encryptionkey@0"
I20260927 07:00:29.523940 16560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260927 07:00:29.528470 16575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:29.530534 16560 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 07:00:29.530741 16560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "1da2d1e2148d4e4eae1fb1ba89543082"
format_stamp: "Formatted at 2026-09-27 07:00:29 on dist-test-slave-p6hz"
server_key: "bffb37ac49e8b502f3af22d3812ce592"
server_key_iv: "5b72c3e85adb0d536b4916165c28782f"
server_key_version: "encryptionkey@0"
I20260927 07:00:29.530982 16560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:29.553387 16560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:29.557637 16560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:29.557916 16560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:29.558894 16560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:29.560266 16560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:29.560343 16560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:29.560456 16560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:29.560503 16560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:29.590523 16560 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:35841
I20260927 07:00:29.590569 16687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:35841 every 8 connection(s)
I20260927 07:00:29.591804 16560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:29.595324 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 16560
I20260927 07:00:29.595543 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:29.596058 13799 external_mini_cluster.cc:1428] Setting key 95d11d8663c29f28d98508f9ab06cfb8
Sep 27 07:00:29 dist-test-slave-p6hz krb5kdc[16187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492429, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:29.613040 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:29.594322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:53119 (local address 127.13.121.254:38093)
0927 07:00:29.594561 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:29.594568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:29.595962 (+ 1394us) server_negotiation.cc:407] Connection header received
0927 07:00:29.598109 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:29.598115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:29.598228 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:29.598312 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:29.600023 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:29.600765 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:29.602567 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:29.602984 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:29.603098 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:29.606812 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:29.606819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:29.606824 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:29.606866 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:29.609692 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:29.610607 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:29.610611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:29.610614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:29.610767 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:29.611316 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:29.611321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:29.611324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:29.611635 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:29.611778 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:29.612564 (+ 786us) server_negotiation.cc:299] Negotiation successful
0927 07:00:29.612750 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 07:00:29.613958 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:29.594839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38093 (local address 127.13.121.195:53119)
0927 07:00:29.595773 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:29.595813 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:29.597795 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:29.598588 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:29.598598 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:29.599233 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:29.599834 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:29.599855 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:29.600940 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:29.600944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:29.602388 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:29.602403 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:29.603261 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:29.603270 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:29.603513 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:29.604159 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:29.604188 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:29.606539 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:29.609828 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:29.609838 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:29.609855 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:29.610477 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:29.610901 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:29.610906 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:29.610909 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:29.611187 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:29.611756 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:29.611763 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:29.612174 (+ 411us) client_negotiation.cc:769] Sending connection context
0927 07:00:29.612421 (+ 247us) client_negotiation.cc:240] Negotiation successful
0927 07:00:29.612823 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":370,"threads_started":1}
I20260927 07:00:29.615484 16688 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38093
I20260927 07:00:29.615795 16688 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:29.616549 16688 heartbeater.cc:511] Master 127.13.121.254:38093 requested a full tablet report, sending...
I20260927 07:00:29.618170 16232 ts_manager.cc:194] Registered new tserver with Master: 1da2d1e2148d4e4eae1fb1ba89543082 (127.13.121.195:35841)
I20260927 07:00:29.619068 16232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:53119
I20260927 07:00:29.624763 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:00:29.639737 16421 heartbeater.cc:503] Master 127.13.121.254:38093 was elected leader, sending a full tablet report...
W20260927 07:00:29.653306 16423 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:00:29.650641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.13.121.254:38093)
0927 07:00:29.650831 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:29.650846 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:29.651231 (+ 385us) server_negotiation.cc:407] Connection header received
0927 07:00:29.651434 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:29.651439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:29.651507 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:29.651601 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:29.653040 (+ 1439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58800: BlockingRecv error: recv got EOF from 127.0.0.1:58800 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 07:00:29.656981 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16293
W20260927 07:00:29.660071 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16293: Not authorized: Client connection negotiation failed: client connection to 127.13.121.193:41557: server requires authentication, but client does not have Kerberos credentials available
I20260927 07:00:29.660290 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 16293
I20260927 07:00:29.673836 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16427
W20260927 07:00:29.676445 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16427: Not authorized: Client connection negotiation failed: client connection to 127.13.121.194:37251: server requires authentication, but client does not have Kerberos credentials available
I20260927 07:00:29.676612 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 16427
I20260927 07:00:29.688444 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16560
W20260927 07:00:29.691340 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16560: Not authorized: Client connection negotiation failed: client connection to 127.13.121.195:35841: server requires authentication, but client does not have Kerberos credentials available
I20260927 07:00:29.691515 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 16560
I20260927 07:00:29.703831 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16200
I20260927 07:00:29.705116 16262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:29.809556 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 16200
2026-09-27T07:00:29Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3438 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:29 dist-test-slave-p6hz krb5kdc[16705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:29 dist-test-slave-p6hz krb5kdc[16705](info): set up 2 sockets
Sep 27 07:00:29 dist-test-slave-p6hz krb5kdc[16705](info): commencing operation
krb5kdc: starting...
W20260927 07:00:30.823624 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.002s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:30 dist-test-slave-p6hz krb5kdc[16705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:31.015291 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:40433
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:40433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:31.342655 16718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:31.342994 16718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:31.343106 16718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:31.350631 16718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:31.350755 16718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:31.350808 16718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:31.350883 16718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:31.350940 16718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:31.361161 16718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:40433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:40433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:31.362490 16718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:31.363860 16718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:31.375563 16724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:31.375777 16723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:31.376047 16726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:31.376439 16718 server_base.cc:1034] running on GCE node
I20260927 07:00:31.377295 16718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:31.378896 16718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:31.380151 16718 hybrid_clock.cc:648] HybridClock initialized: now 1790492431380090 us; error 69 us; skew 500 ppm
I20260927 07:00:31.380548 16718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:31 dist-test-slave-p6hz krb5kdc[16705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492431, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:31.383847 16718 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:31.385327 16718 webserver.cc:460] Webserver started at http://127.13.121.254:42009/ using document root <none> and password file <none>
I20260927 07:00:31.385982 16718 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:31.386080 16718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:31.386358 16718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:31.389328 16718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "f182ce5708944595acafd99a6db27a84"
format_stamp: "Formatted at 2026-09-27 07:00:31 on dist-test-slave-p6hz"
server_key: "48118657616c70e584043956c86f7d99"
server_key_iv: "1abdc1124a08bbf571d3559194e8c0a5"
server_key_version: "encryptionkey@0"
I20260927 07:00:31.390077 16718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "f182ce5708944595acafd99a6db27a84"
format_stamp: "Formatted at 2026-09-27 07:00:31 on dist-test-slave-p6hz"
server_key: "48118657616c70e584043956c86f7d99"
server_key_iv: "1abdc1124a08bbf571d3559194e8c0a5"
server_key_version: "encryptionkey@0"
I20260927 07:00:31.396238 16718 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260927 07:00:31.400106 16733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:31.401729 16718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:31.401947 16718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "f182ce5708944595acafd99a6db27a84"
format_stamp: "Formatted at 2026-09-27 07:00:31 on dist-test-slave-p6hz"
server_key: "48118657616c70e584043956c86f7d99"
server_key_iv: "1abdc1124a08bbf571d3559194e8c0a5"
server_key_version: "encryptionkey@0"
I20260927 07:00:31.402160 16718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:31.426558 16718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:31.431314 16718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:31.431620 16718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:31.449467 16718 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:40433
I20260927 07:00:31.449502 16784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:40433 every 8 connection(s)
I20260927 07:00:31.450812 16718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:31.453979 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 16718
I20260927 07:00:31.454211 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:31.454981 13799 external_mini_cluster.cc:1428] Setting key 623bac7d4b465acfae2e137ce24557b3
I20260927 07:00:31.455055 16785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:31.469650 16785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84: Bootstrap starting.
Sep 27 07:00:31 dist-test-slave-p6hz krb5kdc[16705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:31.473171 16785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:31.474197 16785 log.cc:824] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:31.477383 16785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84: No bootstrap required, opened a new log
I20260927 07:00:31.478984 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:31.456869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33586 (local address 127.13.121.254:40433)
0927 07:00:31.457739 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:31.457753 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:31.457799 (+ 46us) server_negotiation.cc:407] Connection header received
0927 07:00:31.459889 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:31.459918 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:31.460415 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:31.460793 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:31.462640 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:31.464065 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:31.465920 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:31.466735 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:31.466920 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:31.471511 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:31.471520 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:31.471536 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:31.471580 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:31.474512 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:31.475262 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:31.475268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:31.475275 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:31.475485 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:31.475863 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:31.475868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:31.475871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:31.476555 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:31.476844 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:31.477180 (+ 336us) server_negotiation.cc:299] Negotiation successful
0927 07:00:31.477606 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":409,"threads_started":1}
I20260927 07:00:31.483247 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:00:31.485919 16785 raft_consensus.cc:348] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f182ce5708944595acafd99a6db27a84" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40433 } }
I20260927 07:00:31.486290 16785 raft_consensus.cc:374] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:31.486382 16785 raft_consensus.cc:729] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f182ce5708944595acafd99a6db27a84, State: Initialized, Role: FOLLOWER
I20260927 07:00:31.487048 16785 consensus_queue.cc:260] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f182ce5708944595acafd99a6db27a84" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40433 } }
I20260927 07:00:31.487339 16785 raft_consensus.cc:388] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:31.487474 16785 raft_consensus.cc:482] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:31.487651 16785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:31.489799 16785 raft_consensus.cc:504] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f182ce5708944595acafd99a6db27a84" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40433 } }
I20260927 07:00:31.490319 16785 leader_election.cc:302] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f182ce5708944595acafd99a6db27a84; no voters:
I20260927 07:00:31.490934 16785 leader_election.cc:288] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:31.491080 16791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:31.492952 16785 sys_catalog.cc:564] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:31.497581 16791 raft_consensus.cc:686] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [term 1 LEADER]: Becoming Leader. State: Replica: f182ce5708944595acafd99a6db27a84, State: Running, Role: LEADER
I20260927 07:00:31.498212 16791 consensus_queue.cc:237] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f182ce5708944595acafd99a6db27a84" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40433 } }
I20260927 07:00:31.501542 16792 sys_catalog.cc:455] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f182ce5708944595acafd99a6db27a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f182ce5708944595acafd99a6db27a84" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40433 } } }
I20260927 07:00:31.501936 16792 sys_catalog.cc:458] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:31.502740 16793 sys_catalog.cc:455] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f182ce5708944595acafd99a6db27a84. Latest consensus state: current_term: 1 leader_uuid: "f182ce5708944595acafd99a6db27a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f182ce5708944595acafd99a6db27a84" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40433 } } }
I20260927 07:00:31.503665 16793 sys_catalog.cc:458] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84 [sys.catalog]: This master's current role is: LEADER
W20260927 07:00:31.508423 16806 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:00:31.508543 16806 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:00:31.508680 16805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:31.510293 16805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:31.519745 16805 catalog_manager.cc:1347] Generated new cluster ID: 32262af0b71b44f5b2faf5479c670e0e
I20260927 07:00:31.519871 16805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:31.529633 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:31.520221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.13.121.254:40433)
0927 07:00:31.520391 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:31.520396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:31.520985 (+ 589us) server_negotiation.cc:407] Connection header received
0927 07:00:31.521172 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:31.521178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:31.521263 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:31.521380 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:31.522568 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:31.523807 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:31.525636 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:31.526328 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:31.526482 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:31.527770 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:31.527777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:31.527782 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:31.528695 (+ 913us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:31.528766 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:31.528806 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:31.528928 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:31.529128 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 07:00:31.529241 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:00:31.558161 16805 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:31.560199 16805 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:31.580281 16805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f182ce5708944595acafd99a6db27a84: Generated new TSK 0
I20260927 07:00:31.581218 16805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:00:31.590287 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16718
I20260927 07:00:31.591595 16780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:31.683789 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 16718
2026-09-27T07:00:31Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1867 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:31 dist-test-slave-p6hz krb5kdc[16819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:31 dist-test-slave-p6hz krb5kdc[16819](info): set up 2 sockets
Sep 27 07:00:31 dist-test-slave-p6hz krb5kdc[16819](info): commencing operation
krb5kdc: starting...
W20260927 07:00:32.687446 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:32 dist-test-slave-p6hz krb5kdc[16819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:32.870143 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:36057
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:36057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:33.193436 16832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:33.193775 16832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:33.193867 16832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:33.201723 16832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:33.201864 16832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:33.201910 16832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:33.201967 16832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:33.202008 16832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:33.212299 16832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:36057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:36057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:33.213639 16832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:33.214969 16832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:33.227406 16837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:33.227311 16840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:33.227399 16838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:33.228200 16832 server_base.cc:1034] running on GCE node
I20260927 07:00:33.229043 16832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:33.230599 16832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:33.231824 16832 hybrid_clock.cc:648] HybridClock initialized: now 1790492433231755 us; error 60 us; skew 500 ppm
I20260927 07:00:33.232203 16832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:33 dist-test-slave-p6hz krb5kdc[16819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492433, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:33.240114 16832 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:33.241694 16832 webserver.cc:460] Webserver started at http://127.13.121.254:45589/ using document root <none> and password file <none>
I20260927 07:00:33.242460 16832 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:33.242589 16832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:33.242919 16832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:33.246048 16832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "0b9b73e3dd894a96af9b9fd0f973ea73"
format_stamp: "Formatted at 2026-09-27 07:00:33 on dist-test-slave-p6hz"
server_key: "521aa176abd3897f189bb8750ac63491"
server_key_iv: "a0cf05f79e5241ff919e06c1f04223fb"
server_key_version: "encryptionkey@0"
I20260927 07:00:33.246874 16832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "0b9b73e3dd894a96af9b9fd0f973ea73"
format_stamp: "Formatted at 2026-09-27 07:00:33 on dist-test-slave-p6hz"
server_key: "521aa176abd3897f189bb8750ac63491"
server_key_iv: "a0cf05f79e5241ff919e06c1f04223fb"
server_key_version: "encryptionkey@0"
I20260927 07:00:33.252576 16832 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260927 07:00:33.256984 16847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:33.259114 16832 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 07:00:33.259397 16832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "0b9b73e3dd894a96af9b9fd0f973ea73"
format_stamp: "Formatted at 2026-09-27 07:00:33 on dist-test-slave-p6hz"
server_key: "521aa176abd3897f189bb8750ac63491"
server_key_iv: "a0cf05f79e5241ff919e06c1f04223fb"
server_key_version: "encryptionkey@0"
I20260927 07:00:33.259625 16832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:33.282402 16832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:33.286701 16832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:33.286979 16832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:33.303887 16832 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:36057
I20260927 07:00:33.303931 16898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:36057 every 8 connection(s)
I20260927 07:00:33.305218 16832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:33.307963 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 16832
I20260927 07:00:33.308136 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:33.308738 13799 external_mini_cluster.cc:1428] Setting key 78308b5c81f9a35532b1925f20ec1ebb
I20260927 07:00:33.308936 16899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:33.322861 16899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73: Bootstrap starting.
Sep 27 07:00:33 dist-test-slave-p6hz krb5kdc[16819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:33.325965 16899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:33.327016 16899 log.cc:824] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:33.329888 16899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73: No bootstrap required, opened a new log
I20260927 07:00:33.330837 16902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:33.310554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.13.121.254:36057)
0927 07:00:33.311313 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:33.311324 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:33.311369 (+ 45us) server_negotiation.cc:407] Connection header received
0927 07:00:33.313277 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:33.313299 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:33.313714 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:33.314057 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:33.315752 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:33.317061 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:33.318708 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:33.319472 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:33.319619 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:33.324017 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:33.324026 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:33.324040 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:33.324090 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:33.326925 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:33.327618 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:33.327625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:33.327631 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:33.327822 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:33.328183 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:33.328187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:33.328189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:33.328690 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:33.328969 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:33.329407 (+ 438us) server_negotiation.cc:299] Negotiation successful
0927 07:00:33.329808 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":327,"threads_started":1}
I20260927 07:00:33.337641 16899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9b73e3dd894a96af9b9fd0f973ea73" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36057 } }
I20260927 07:00:33.338012 16899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:33.338099 16899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b9b73e3dd894a96af9b9fd0f973ea73, State: Initialized, Role: FOLLOWER
I20260927 07:00:33.338627 16899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9b73e3dd894a96af9b9fd0f973ea73" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36057 } }
I20260927 07:00:33.338830 16899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:33.338948 16899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:33.339087 16899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:33.341343 16899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9b73e3dd894a96af9b9fd0f973ea73" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36057 } }
I20260927 07:00:33.341804 16899 leader_election.cc:302] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b9b73e3dd894a96af9b9fd0f973ea73; no voters:
I20260927 07:00:33.342394 16899 leader_election.cc:288] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:33.342497 16904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:33.343184 16904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [term 1 LEADER]: Becoming Leader. State: Replica: 0b9b73e3dd894a96af9b9fd0f973ea73, State: Running, Role: LEADER
I20260927 07:00:33.343737 16904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9b73e3dd894a96af9b9fd0f973ea73" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36057 } }
I20260927 07:00:33.344660 16899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:33.348603 16906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b9b73e3dd894a96af9b9fd0f973ea73. Latest consensus state: current_term: 1 leader_uuid: "0b9b73e3dd894a96af9b9fd0f973ea73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9b73e3dd894a96af9b9fd0f973ea73" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36057 } } }
I20260927 07:00:33.348655 16905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b9b73e3dd894a96af9b9fd0f973ea73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9b73e3dd894a96af9b9fd0f973ea73" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36057 } } }
I20260927 07:00:33.348971 16906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:33.349000 16905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:33.354631 16915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:33.358897 16915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:33.367975 16915 catalog_manager.cc:1347] Generated new cluster ID: 9e48cf3dab2f422eb0169ff66e657ae5
I20260927 07:00:33.368098 16915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:33.379211 16915 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:33.380581 16915 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:33.385896 16915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b9b73e3dd894a96af9b9fd0f973ea73: Generated new TSK 0
I20260927 07:00:33.387007 16915 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:00:33.430891 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36057
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:33.754279 16925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:33.754580 16925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:33.754700 16925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:33.762364 16925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:33.762482 16925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:33.762643 16925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:33.773007 16925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:33.774335 16925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:33.775661 16925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:33.789166 16931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:33.789099 16930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:33.789131 16933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:33.789907 16925 server_base.cc:1034] running on GCE node
I20260927 07:00:33.790521 16925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:33.791498 16925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:33.792685 16925 hybrid_clock.cc:648] HybridClock initialized: now 1790492433792630 us; error 52 us; skew 500 ppm
I20260927 07:00:33.792989 16925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:33 dist-test-slave-p6hz krb5kdc[16819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492433, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:33.796263 16925 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:00:33.797770 16925 webserver.cc:460] Webserver started at http://127.13.121.193:36953/ using document root <none> and password file <none>
I20260927 07:00:33.798480 16925 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:33.798595 16925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:33.798902 16925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:33.801950 16925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "33cf9986f6694d3eb5d92f766ddbdddf"
format_stamp: "Formatted at 2026-09-27 07:00:33 on dist-test-slave-p6hz"
server_key: "9f4ac43e011e0df64fa1349981faca7c"
server_key_iv: "f1d815be168f141e2fe05287bdc996a3"
server_key_version: "encryptionkey@0"
I20260927 07:00:33.802764 16925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "33cf9986f6694d3eb5d92f766ddbdddf"
format_stamp: "Formatted at 2026-09-27 07:00:33 on dist-test-slave-p6hz"
server_key: "9f4ac43e011e0df64fa1349981faca7c"
server_key_iv: "f1d815be168f141e2fe05287bdc996a3"
server_key_version: "encryptionkey@0"
I20260927 07:00:33.808449 16925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260927 07:00:33.812112 16940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:33.813558 16925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260927 07:00:33.813730 16925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "33cf9986f6694d3eb5d92f766ddbdddf"
format_stamp: "Formatted at 2026-09-27 07:00:33 on dist-test-slave-p6hz"
server_key: "9f4ac43e011e0df64fa1349981faca7c"
server_key_iv: "f1d815be168f141e2fe05287bdc996a3"
server_key_version: "encryptionkey@0"
I20260927 07:00:33.813915 16925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:33.827425 16925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:33.831979 16925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:33.832273 16925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:33.833247 16925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:33.834617 16925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:33.834697 16925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:33.834813 16925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:33.834864 16925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:33.864058 16925 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:41455
I20260927 07:00:33.864075 17052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:41455 every 8 connection(s)
I20260927 07:00:33.865427 16925 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:33.868521 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 16925
I20260927 07:00:33.868721 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:33.869490 13799 external_mini_cluster.cc:1428] Setting key b560ee142b3427dc658b1eb3abd0e056
Sep 27 07:00:33 dist-test-slave-p6hz krb5kdc[16819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492433, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:00:33.888064 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:33.868043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:45141 (local address 127.13.121.254:36057)
0927 07:00:33.868535 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:33.868541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:33.869641 (+ 1100us) server_negotiation.cc:407] Connection header received
0927 07:00:33.871677 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:33.871680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:33.871745 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:33.871822 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:33.874027 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:33.874751 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:33.877228 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:33.877847 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:33.877981 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:33.881516 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:33.881522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:33.881527 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:33.881567 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:33.884711 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:33.885632 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:33.885637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:33.885644 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:33.885774 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:33.886287 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:33.886291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:33.886293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:33.886570 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:33.886699 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:33.887467 (+ 768us) server_negotiation.cc:299] Negotiation successful
0927 07:00:33.887697 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":242,"threads_started":1}
I20260927 07:00:33.889174 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:33.868619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36057 (local address 127.13.121.193:45141)
0927 07:00:33.869453 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:33.869507 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:33.871357 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:33.872173 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:33.872185 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:33.872949 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:33.873816 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:33.873845 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:33.874938 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:33.874944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:33.877063 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:33.877081 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:33.878079 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:33.878084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:33.878315 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:33.878966 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:33.878991 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:33.881253 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:33.884929 (+ 3676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:33.884938 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:33.884955 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:33.885457 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:33.885893 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:33.885897 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:33.885901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:33.886172 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:33.886693 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:33.886702 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:33.887059 (+ 357us) client_negotiation.cc:769] Sending connection context
0927 07:00:33.887356 (+ 297us) client_negotiation.cc:240] Negotiation successful
0927 07:00:33.887930 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":286,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
I20260927 07:00:33.890713 17053 heartbeater.cc:347] Connected to a master server at 127.13.121.254:36057
I20260927 07:00:33.891047 17053 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:33.891788 17053 heartbeater.cc:511] Master 127.13.121.254:36057 requested a full tablet report, sending...
I20260927 07:00:33.893873 16864 ts_manager.cc:194] Registered new tserver with Master: 33cf9986f6694d3eb5d92f766ddbdddf (127.13.121.193:41455)
I20260927 07:00:33.895690 16864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:45141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:00:33.914359 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36057
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:34.238534 17059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:34.238850 17059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:34.238970 17059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:34.246493 17059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:34.246613 17059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:34.246764 17059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:34.256703 17059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:34.258040 17059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:34.259409 17059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:34.272324 17065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:34.272352 17067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:34.272567 17064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:34.273005 17059 server_base.cc:1034] running on GCE node
I20260927 07:00:34.273629 17059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:34.274567 17059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:34.275763 17059 hybrid_clock.cc:648] HybridClock initialized: now 1790492434275714 us; error 38 us; skew 500 ppm
I20260927 07:00:34.276101 17059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:34 dist-test-slave-p6hz krb5kdc[16819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492434, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:34.279042 17059 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:00:34.280381 17059 webserver.cc:460] Webserver started at http://127.13.121.194:45111/ using document root <none> and password file <none>
I20260927 07:00:34.281044 17059 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:34.281160 17059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:34.281438 17059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:34.284349 17059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "cb4ccc095bf5494fa291a82f2fe2020a"
format_stamp: "Formatted at 2026-09-27 07:00:34 on dist-test-slave-p6hz"
server_key: "6bcbf3c7d4893e48e4972a5bc36ff7d6"
server_key_iv: "d073913ece1374493f1eeda269034012"
server_key_version: "encryptionkey@0"
I20260927 07:00:34.285089 17059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "cb4ccc095bf5494fa291a82f2fe2020a"
format_stamp: "Formatted at 2026-09-27 07:00:34 on dist-test-slave-p6hz"
server_key: "6bcbf3c7d4893e48e4972a5bc36ff7d6"
server_key_iv: "d073913ece1374493f1eeda269034012"
server_key_version: "encryptionkey@0"
I20260927 07:00:34.290064 17059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:00:34.293648 17074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:34.295073 17059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260927 07:00:34.295255 17059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "cb4ccc095bf5494fa291a82f2fe2020a"
format_stamp: "Formatted at 2026-09-27 07:00:34 on dist-test-slave-p6hz"
server_key: "6bcbf3c7d4893e48e4972a5bc36ff7d6"
server_key_iv: "d073913ece1374493f1eeda269034012"
server_key_version: "encryptionkey@0"
I20260927 07:00:34.295441 17059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:34.313555 17059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:34.318707 17059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:34.319036 17059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:34.320195 17059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:34.321844 17059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:34.321935 17059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:34.322049 17059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:34.322108 17059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:34.352720 17059 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:43867
I20260927 07:00:34.352837 17186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:43867 every 8 connection(s)
I20260927 07:00:34.353982 17059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:34.362875 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 17059
I20260927 07:00:34.363049 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:34.363549 13799 external_mini_cluster.cc:1428] Setting key 41e1d9edfea31462cebd0071e945ddfc
Sep 27 07:00:34 dist-test-slave-p6hz krb5kdc[16819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492434, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:34.374351 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:34.356480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:36017 (local address 127.13.121.254:36057)
0927 07:00:34.356676 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:34.356680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:34.357808 (+ 1128us) server_negotiation.cc:407] Connection header received
0927 07:00:34.359558 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:34.359562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:34.359632 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:34.359719 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:34.361163 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.361876 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.363493 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.364150 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.364248 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:34.368203 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:34.368209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:34.368213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:34.368271 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:34.371174 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.372149 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.372154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.372156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.372305 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.372749 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.372752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.372754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.373015 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:34.373127 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:34.373911 (+ 784us) server_negotiation.cc:299] Negotiation successful
0927 07:00:34.374091 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260927 07:00:34.375806 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:34.356953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36057 (local address 127.13.121.194:36017)
0927 07:00:34.357657 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:34.357699 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:34.359297 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:34.359942 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:34.359950 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:34.360470 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:34.361017 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:34.361033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.361989 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.361992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:34.363364 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:34.363375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.364369 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.364375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:34.364658 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:34.365410 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:34.365434 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:34.367970 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:34.371351 (+ 3381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:34.371361 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:34.371382 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:34.371995 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:34.372409 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:34.372414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:34.372417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:34.372658 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:34.373128 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:34.373135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:34.373559 (+ 424us) client_negotiation.cc:769] Sending connection context
0927 07:00:34.373843 (+ 284us) client_negotiation.cc:240] Negotiation successful
0927 07:00:34.374393 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":266,"threads_started":1}
I20260927 07:00:34.377346 17187 heartbeater.cc:347] Connected to a master server at 127.13.121.254:36057
I20260927 07:00:34.377647 17187 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:34.378432 17187 heartbeater.cc:511] Master 127.13.121.254:36057 requested a full tablet report, sending...
I20260927 07:00:34.379987 16864 ts_manager.cc:194] Registered new tserver with Master: cb4ccc095bf5494fa291a82f2fe2020a (127.13.121.194:43867)
I20260927 07:00:34.381124 16864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:36017
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:00:34.406206 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36057
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:34.722306 17192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:34.722599 17192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:34.722731 17192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:34.730017 17192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:34.730141 17192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:34.730280 17192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:34.740826 17192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:34.742144 17192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:34.743523 17192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:34.757380 17197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:34.757433 17198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:34.757339 17200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:34.758313 17192 server_base.cc:1034] running on GCE node
I20260927 07:00:34.759006 17192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:34.760080 17192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:34.761312 17192 hybrid_clock.cc:648] HybridClock initialized: now 1790492434761235 us; error 63 us; skew 500 ppm
I20260927 07:00:34.761636 17192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:34 dist-test-slave-p6hz krb5kdc[16819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492434, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:34.764885 17192 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:00:34.766362 17192 webserver.cc:460] Webserver started at http://127.13.121.195:43479/ using document root <none> and password file <none>
I20260927 07:00:34.767059 17192 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:34.767200 17192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:34.767490 17192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:34.770376 17192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "c11d3814be6840dd9855692930baf27c"
format_stamp: "Formatted at 2026-09-27 07:00:34 on dist-test-slave-p6hz"
server_key: "ab5a22b876edf81982453ffd4c855886"
server_key_iv: "8718b6974fab2ecc0e6a2185730a20b3"
server_key_version: "encryptionkey@0"
I20260927 07:00:34.771173 17192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "c11d3814be6840dd9855692930baf27c"
format_stamp: "Formatted at 2026-09-27 07:00:34 on dist-test-slave-p6hz"
server_key: "ab5a22b876edf81982453ffd4c855886"
server_key_iv: "8718b6974fab2ecc0e6a2185730a20b3"
server_key_version: "encryptionkey@0"
I20260927 07:00:34.776377 17192 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260927 07:00:34.780053 17207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:34.781430 17192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:34.781594 17192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "c11d3814be6840dd9855692930baf27c"
format_stamp: "Formatted at 2026-09-27 07:00:34 on dist-test-slave-p6hz"
server_key: "ab5a22b876edf81982453ffd4c855886"
server_key_iv: "8718b6974fab2ecc0e6a2185730a20b3"
server_key_version: "encryptionkey@0"
I20260927 07:00:34.781811 17192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:34.802479 17192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:34.806612 17192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:34.806888 17192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:34.807873 17192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:34.809221 17192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:34.809300 17192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:34.809424 17192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:34.809475 17192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:34.839268 17192 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:45357
I20260927 07:00:34.839286 17319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:45357 every 8 connection(s)
I20260927 07:00:34.840531 17192 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:34.843168 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 17192
I20260927 07:00:34.843335 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:34.843904 13799 external_mini_cluster.cc:1428] Setting key 817008925cc7d233a86f15d766af72ac
Sep 27 07:00:34 dist-test-slave-p6hz krb5kdc[16819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492434, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:34.860334 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:34.842979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:53189 (local address 127.13.121.254:36057)
0927 07:00:34.843176 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:34.843183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:34.844697 (+ 1514us) server_negotiation.cc:407] Connection header received
0927 07:00:34.846811 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:34.846816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:34.846903 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:34.846990 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:34.848528 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.849267 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.850756 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.851174 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.851280 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:34.854651 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:34.854656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:34.854660 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:34.854693 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:34.857021 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.857900 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.857905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.857909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.858043 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.858535 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.858541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.858544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.858884 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:34.859056 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:34.859902 (+ 846us) server_negotiation.cc:299] Negotiation successful
0927 07:00:34.860077 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 07:00:34.861325 17322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:34.843680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36057 (local address 127.13.121.195:53189)
0927 07:00:34.844477 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:34.844532 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:34.846495 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:34.847263 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:34.847272 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:34.847778 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:34.848379 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:34.848397 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.849398 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.849401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:34.850621 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:34.850633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.851416 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.851421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:34.851635 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:34.852247 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:34.852275 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:34.854442 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:34.857165 (+ 2723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:34.857172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:34.857186 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:34.857766 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:34.858152 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:34.858156 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:34.858160 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:34.858379 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:34.859049 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:34.859058 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:34.859514 (+ 456us) client_negotiation.cc:769] Sending connection context
0927 07:00:34.859828 (+ 314us) client_negotiation.cc:240] Negotiation successful
0927 07:00:34.860257 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":311,"threads_started":1}
I20260927 07:00:34.862612 17320 heartbeater.cc:347] Connected to a master server at 127.13.121.254:36057
I20260927 07:00:34.862895 17320 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:34.863729 17320 heartbeater.cc:511] Master 127.13.121.254:36057 requested a full tablet report, sending...
I20260927 07:00:34.865181 16864 ts_manager.cc:194] Registered new tserver with Master: c11d3814be6840dd9855692930baf27c (127.13.121.195:45357)
I20260927 07:00:34.865950 16864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:53189
I20260927 07:00:34.872088 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:00:34 dist-test-slave-p6hz krb5kdc[16819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492434, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
I20260927 07:00:34.899391 17053 heartbeater.cc:503] Master 127.13.121.254:36057 was elected leader, sending a full tablet report...
Sep 27 07:00:34 dist-test-slave-p6hz krb5kdc[16819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492434, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:34.916812 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:34.903004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34658 (local address 127.13.121.254:36057)
0927 07:00:34.903187 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:34.903192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:34.903708 (+ 516us) server_negotiation.cc:407] Connection header received
0927 07:00:34.903809 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:34.903813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:34.903877 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:34.903965 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:34.905262 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.906036 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.907351 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.907918 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.908049 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:34.911854 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:34.911859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:34.911863 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:34.911901 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:34.914235 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.914960 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.914964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.914967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.915221 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.915616 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.915619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.915621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.915950 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:34.916124 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:34.916411 (+ 287us) server_negotiation.cc:299] Negotiation successful
0927 07:00:34.916583 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260927 07:00:34.917352 16864 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34658
I20260927 07:00:34.919039 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16925
Sep 27 07:00:34 dist-test-slave-p6hz krb5kdc[16819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492434, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:34.935767 17331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:34.920556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50400 (local address 127.13.121.193:41455)
0927 07:00:34.921231 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:34.921239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:34.921265 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:00:34.921321 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:34.921328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:34.921538 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:34.921660 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:34.922876 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.923969 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.925222 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.925880 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.925989 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:34.929737 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:34.929751 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:34.929768 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:34.929847 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:34.932810 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.933506 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.933514 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.933516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.933645 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.933987 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.933991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.933993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.934381 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:34.934610 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:34.935093 (+ 483us) server_negotiation.cc:299] Negotiation successful
0927 07:00:34.935471 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":366,"threads_started":1}
W20260927 07:00:34.937048 17048 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50400
W20260927 07:00:34.938108 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16925: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:00:34.938299 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 16925
I20260927 07:00:34.950742 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17059
Sep 27 07:00:34 dist-test-slave-p6hz krb5kdc[16819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492434, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:00:34.966079 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:34.951590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.13.121.194:43867)
0927 07:00:34.952092 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:34.952099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:34.952125 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:00:34.952182 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:34.952188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:34.952401 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:34.952522 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:34.953711 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.954716 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.956039 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.956612 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.956719 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:34.960544 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:34.960557 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:34.960572 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:34.960611 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:34.963329 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.964042 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.964049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.964052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.964175 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.964539 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.964544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.964546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.964966 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:34.965206 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:34.965451 (+ 245us) server_negotiation.cc:299] Negotiation successful
0927 07:00:34.965667 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":262,"threads_started":1}
W20260927 07:00:34.967362 17182 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38598
W20260927 07:00:34.968407 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17059: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:00:34.968549 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 17059
I20260927 07:00:34.980463 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17192
Sep 27 07:00:34 dist-test-slave-p6hz krb5kdc[16819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492434, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:00:34.995481 17333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:34.981391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39206 (local address 127.13.121.195:45357)
0927 07:00:34.981987 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:34.981995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:34.982023 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:00:34.982082 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:34.982088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:34.982285 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:34.982406 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:34.983704 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.984690 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.985875 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:34.986463 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:34.986568 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:34.990463 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:34.990474 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:34.990488 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:34.990529 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:34.993152 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.993729 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.993741 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.993744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.993858 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:34.994142 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:34.994145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:34.994147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:34.994478 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:34.994686 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:34.994884 (+ 198us) server_negotiation.cc:299] Negotiation successful
0927 07:00:34.995076 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":292,"threads_started":1}
W20260927 07:00:34.996730 17315 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39206
W20260927 07:00:34.997808 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17192: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:00:34.997983 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 17192
I20260927 07:00:35.009959 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 16832
I20260927 07:00:35.011127 16894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:35.111582 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 16832
2026-09-27T07:00:35Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3432 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:35 dist-test-slave-p6hz krb5kdc[17337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:35 dist-test-slave-p6hz krb5kdc[17337](info): set up 2 sockets
Sep 27 07:00:35 dist-test-slave-p6hz krb5kdc[17337](info): commencing operation
krb5kdc: starting...
W20260927 07:00:36.115553 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.002s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:36 dist-test-slave-p6hz krb5kdc[17337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:36.300500 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:39751
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:39751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:36.623637 17350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:36.623951 17350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:36.624070 17350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:36.631397 17350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:36.631508 17350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:36.631557 17350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:36.631633 17350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:36.631687 17350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:36.641765 17350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:39751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:39751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:36.643247 17350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:36.644644 17350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:36.657112 17355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:36.657080 17358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:36.657047 17356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:36.657902 17350 server_base.cc:1034] running on GCE node
I20260927 07:00:36.658722 17350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:36.660332 17350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:36.661563 17350 hybrid_clock.cc:648] HybridClock initialized: now 1790492436661509 us; error 43 us; skew 500 ppm
I20260927 07:00:36.661962 17350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:36 dist-test-slave-p6hz krb5kdc[17337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492436, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:36.665084 17350 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:36.666673 17350 webserver.cc:460] Webserver started at http://127.13.121.254:40025/ using document root <none> and password file <none>
I20260927 07:00:36.667429 17350 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:36.667547 17350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:36.667872 17350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:36.670909 17350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "e63f83d73f544c9f8e26953df075baf5"
format_stamp: "Formatted at 2026-09-27 07:00:36 on dist-test-slave-p6hz"
server_key: "849a712081cac50b95cd216929519044"
server_key_iv: "3c3a725c7cd7030c0f0734dc328821af"
server_key_version: "encryptionkey@0"
I20260927 07:00:36.671777 17350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "e63f83d73f544c9f8e26953df075baf5"
format_stamp: "Formatted at 2026-09-27 07:00:36 on dist-test-slave-p6hz"
server_key: "849a712081cac50b95cd216929519044"
server_key_iv: "3c3a725c7cd7030c0f0734dc328821af"
server_key_version: "encryptionkey@0"
I20260927 07:00:36.677246 17350 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:00:36.680900 17365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:36.682421 17350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:00:36.682621 17350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "e63f83d73f544c9f8e26953df075baf5"
format_stamp: "Formatted at 2026-09-27 07:00:36 on dist-test-slave-p6hz"
server_key: "849a712081cac50b95cd216929519044"
server_key_iv: "3c3a725c7cd7030c0f0734dc328821af"
server_key_version: "encryptionkey@0"
I20260927 07:00:36.682871 17350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:36.704124 17350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:36.708256 17350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:36.708531 17350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:36.725730 17350 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:39751
I20260927 07:00:36.725759 17416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:39751 every 8 connection(s)
I20260927 07:00:36.727098 17350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:36.728096 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 17350
I20260927 07:00:36.728328 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:36.729079 13799 external_mini_cluster.cc:1428] Setting key aeb05b0aabe0ef21bfe70b43037bba6e
I20260927 07:00:36.731912 17417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 07:00:36 dist-test-slave-p6hz krb5kdc[17337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:36.745561 17417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5: Bootstrap starting.
I20260927 07:00:36.749807 17417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:36.751282 17417 log.cc:824] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:36.754024 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:36.731075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.13.121.254:39751)
0927 07:00:36.731857 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:36.731869 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:36.731912 (+ 43us) server_negotiation.cc:407] Connection header received
0927 07:00:36.733513 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:36.733535 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:36.733929 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:36.734263 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:36.736020 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:36.737453 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:36.739341 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:36.740037 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:36.740187 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:36.745025 (+ 4838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:36.745042 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:36.745064 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:36.745118 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:36.748993 (+ 3875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:36.749891 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:36.749902 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:36.749911 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:36.750154 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:36.750591 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:36.750596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:36.750599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:36.751472 (+ 873us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:36.751786 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:36.752200 (+ 414us) server_negotiation.cc:299] Negotiation successful
0927 07:00:36.752613 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":355,"threads_started":1}
I20260927 07:00:36.754891 17417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5: No bootstrap required, opened a new log
I20260927 07:00:36.762902 17417 raft_consensus.cc:348] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63f83d73f544c9f8e26953df075baf5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39751 } }
I20260927 07:00:36.763334 17417 raft_consensus.cc:374] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:36.763419 17417 raft_consensus.cc:729] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63f83d73f544c9f8e26953df075baf5, State: Initialized, Role: FOLLOWER
I20260927 07:00:36.763938 17417 consensus_queue.cc:260] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63f83d73f544c9f8e26953df075baf5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39751 } }
I20260927 07:00:36.764134 17417 raft_consensus.cc:388] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:36.764237 17417 raft_consensus.cc:482] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:36.764366 17417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:36.766474 17417 raft_consensus.cc:504] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63f83d73f544c9f8e26953df075baf5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39751 } }
I20260927 07:00:36.766952 17417 leader_election.cc:302] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e63f83d73f544c9f8e26953df075baf5; no voters:
I20260927 07:00:36.767558 17417 leader_election.cc:288] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:36.767691 17422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:36.768379 17422 raft_consensus.cc:686] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [term 1 LEADER]: Becoming Leader. State: Replica: e63f83d73f544c9f8e26953df075baf5, State: Running, Role: LEADER
I20260927 07:00:36.768931 17422 consensus_queue.cc:237] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63f83d73f544c9f8e26953df075baf5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39751 } }
I20260927 07:00:36.769577 17417 sys_catalog.cc:564] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:36.772331 17424 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e63f83d73f544c9f8e26953df075baf5. Latest consensus state: current_term: 1 leader_uuid: "e63f83d73f544c9f8e26953df075baf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63f83d73f544c9f8e26953df075baf5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39751 } } }
I20260927 07:00:36.772807 17424 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:36.772338 17423 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e63f83d73f544c9f8e26953df075baf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63f83d73f544c9f8e26953df075baf5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39751 } } }
I20260927 07:00:36.773168 17423 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:36.783910 17429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:36.789469 17429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:36.798813 17429 catalog_manager.cc:1347] Generated new cluster ID: 994fb2cdd06d43968db290b25a357cd0
I20260927 07:00:36.798949 17429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:36.816340 17429 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:36.818269 17429 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:36.833022 17429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e63f83d73f544c9f8e26953df075baf5: Generated new TSK 0
I20260927 07:00:36.834192 17429 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:00:36.891165 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39751
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:37.226460 17443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:37.226784 17443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:37.226912 17443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:37.234654 17443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:37.234828 17443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:37.235023 17443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:37.245203 17443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:37.246549 17443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:37.247923 17443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:37.261703 17448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:37.261804 17451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:37.261727 17449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:37.262444 17443 server_base.cc:1034] running on GCE node
I20260927 07:00:37.263057 17443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:37.264132 17443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:37.265359 17443 hybrid_clock.cc:648] HybridClock initialized: now 1790492437265286 us; error 65 us; skew 500 ppm
I20260927 07:00:37.265697 17443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:37 dist-test-slave-p6hz krb5kdc[17337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492437, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:37.268962 17443 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:00:37.270630 17443 webserver.cc:460] Webserver started at http://127.13.121.193:35277/ using document root <none> and password file <none>
I20260927 07:00:37.271380 17443 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:37.271478 17443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:37.271727 17443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:37.274766 17443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "dfff7b55b4b4451d8f09659a1b0ea7d4"
format_stamp: "Formatted at 2026-09-27 07:00:37 on dist-test-slave-p6hz"
server_key: "64335ca430ef90e7564a9f10abf7124b"
server_key_iv: "8c558e5f0361213c5feb8262b5914ee0"
server_key_version: "encryptionkey@0"
I20260927 07:00:37.275599 17443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "dfff7b55b4b4451d8f09659a1b0ea7d4"
format_stamp: "Formatted at 2026-09-27 07:00:37 on dist-test-slave-p6hz"
server_key: "64335ca430ef90e7564a9f10abf7124b"
server_key_iv: "8c558e5f0361213c5feb8262b5914ee0"
server_key_version: "encryptionkey@0"
I20260927 07:00:37.280942 17443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:00:37.284767 17458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:37.286198 17443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:37.286366 17443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "dfff7b55b4b4451d8f09659a1b0ea7d4"
format_stamp: "Formatted at 2026-09-27 07:00:37 on dist-test-slave-p6hz"
server_key: "64335ca430ef90e7564a9f10abf7124b"
server_key_iv: "8c558e5f0361213c5feb8262b5914ee0"
server_key_version: "encryptionkey@0"
I20260927 07:00:37.286581 17443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:37.306871 17443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:37.310882 17443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:37.311200 17443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:37.312227 17443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:37.313591 17443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:37.313664 17443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:37.313776 17443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:37.313839 17443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:37.344223 17443 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:40145
I20260927 07:00:37.344259 17570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:40145 every 8 connection(s)
I20260927 07:00:37.345463 17443 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:37.351432 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 17443
I20260927 07:00:37.351616 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:37.352237 13799 external_mini_cluster.cc:1428] Setting key 4e19768e1ac5bacd7c60b53a81dd3861
Sep 27 07:00:37 dist-test-slave-p6hz krb5kdc[17337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492437, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:37.366919 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:37.348194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:51109 (local address 127.13.121.254:39751)
0927 07:00:37.348669 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:37.348673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:37.349765 (+ 1092us) server_negotiation.cc:407] Connection header received
0927 07:00:37.351714 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:37.351719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:37.351806 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:37.351896 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:37.353715 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:37.354731 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:37.356470 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:37.357007 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:37.357146 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:37.360906 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:37.360913 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:37.360917 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:37.360952 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:37.363491 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:37.364456 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:37.364461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:37.364463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:37.364591 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:37.365083 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:37.365086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:37.365088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:37.365354 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:37.365504 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:37.366476 (+ 972us) server_negotiation.cc:299] Negotiation successful
0927 07:00:37.366656 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":224,"threads_started":1}
I20260927 07:00:37.368463 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:37.348777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39751 (local address 127.13.121.193:51109)
0927 07:00:37.349599 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:37.349638 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:37.351412 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:37.352151 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:37.352160 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:37.352731 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:37.353507 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:37.353533 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:37.354931 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:37.354935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:37.356312 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:37.356326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:37.357234 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:37.357239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:37.357453 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:37.358151 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:37.358181 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:37.360622 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:37.363648 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:37.363658 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:37.363680 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:37.364301 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:37.364694 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:37.364699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:37.364702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:37.364965 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:37.365497 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:37.365506 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:37.366028 (+ 522us) client_negotiation.cc:769] Sending connection context
0927 07:00:37.366358 (+ 330us) client_negotiation.cc:240] Negotiation successful
0927 07:00:37.366965 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":280,"threads_started":1}
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:00:37.370117 17571 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39751
I20260927 07:00:37.370431 17571 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:37.371347 17571 heartbeater.cc:511] Master 127.13.121.254:39751 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
I20260927 07:00:37.373611 17382 ts_manager.cc:194] Registered new tserver with Master: dfff7b55b4b4451d8f09659a1b0ea7d4 (127.13.121.193:40145)
I20260927 07:00:37.375592 17382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:51109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:00:37.398128 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39751
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:37.717233 17577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:37.717510 17577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:37.717598 17577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:37.724898 17577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:37.725041 17577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:37.725193 17577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:37.735334 17577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:37.736653 17577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:37.738080 17577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:37.753229 17583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:37.753062 17582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:37.753346 17585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:37.753993 17577 server_base.cc:1034] running on GCE node
I20260927 07:00:37.754695 17577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:37.755803 17577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:37.757022 17577 hybrid_clock.cc:648] HybridClock initialized: now 1790492437756962 us; error 57 us; skew 500 ppm
I20260927 07:00:37.757359 17577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:37 dist-test-slave-p6hz krb5kdc[17337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492437, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:37.760622 17577 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:00:37.762106 17577 webserver.cc:460] Webserver started at http://127.13.121.194:33231/ using document root <none> and password file <none>
I20260927 07:00:37.762822 17577 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:37.762943 17577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:37.763306 17577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:37.766546 17577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "4377952f654643b5807b6c28da59fa20"
format_stamp: "Formatted at 2026-09-27 07:00:37 on dist-test-slave-p6hz"
server_key: "713e21d98f5b1579891b5788fec1600f"
server_key_iv: "dcf46cf06d2f2a838b98a87089d42888"
server_key_version: "encryptionkey@0"
I20260927 07:00:37.767426 17577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "4377952f654643b5807b6c28da59fa20"
format_stamp: "Formatted at 2026-09-27 07:00:37 on dist-test-slave-p6hz"
server_key: "713e21d98f5b1579891b5788fec1600f"
server_key_iv: "dcf46cf06d2f2a838b98a87089d42888"
server_key_version: "encryptionkey@0"
I20260927 07:00:37.772819 17577 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:00:37.776266 17592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:37.777622 17577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:37.777774 17577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "4377952f654643b5807b6c28da59fa20"
format_stamp: "Formatted at 2026-09-27 07:00:37 on dist-test-slave-p6hz"
server_key: "713e21d98f5b1579891b5788fec1600f"
server_key_iv: "dcf46cf06d2f2a838b98a87089d42888"
server_key_version: "encryptionkey@0"
I20260927 07:00:37.777987 17577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:37.801218 17577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:37.805246 17577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:37.805510 17577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:37.806512 17577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:37.807888 17577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:37.807971 17577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:37.808081 17577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:37.808138 17577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:37.837394 17577 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:38957
I20260927 07:00:37.837445 17704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:38957 every 8 connection(s)
I20260927 07:00:37.838672 17577 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:37.846395 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 17577
I20260927 07:00:37.846565 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:37.846963 13799 external_mini_cluster.cc:1428] Setting key 5b140bf3a5713f53a3317da2d4eb4a25
Sep 27 07:00:37 dist-test-slave-p6hz krb5kdc[17337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492437, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:37.858762 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:37.841247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:51415 (local address 127.13.121.254:39751)
0927 07:00:37.841429 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:37.841433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:37.842610 (+ 1177us) server_negotiation.cc:407] Connection header received
0927 07:00:37.844399 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:37.844405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:37.844489 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:37.844573 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:37.846089 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:37.846859 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:37.848603 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:37.849102 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:37.849233 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:37.852941 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:37.852949 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:37.852956 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:37.853000 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:37.855549 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:37.856439 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:37.856446 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:37.856449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:37.856596 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:37.857035 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:37.857039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:37.857041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:37.857301 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:37.857436 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:37.858187 (+ 751us) server_negotiation.cc:299] Negotiation successful
0927 07:00:37.858430 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:00:37.859858 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:37.841737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39751 (local address 127.13.121.194:51415)
0927 07:00:37.842445 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:37.842491 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:37.844118 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:37.844819 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:37.844828 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:37.845347 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:37.845899 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:37.845917 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:37.846995 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:37.847000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:37.848463 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:37.848476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:37.849362 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:37.849368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:37.849576 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:37.850257 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:37.850281 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:37.852648 (+ 2367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:37.855713 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:37.855721 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:37.855746 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:37.856233 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:37.856712 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:37.856716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:37.856718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:37.856918 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:37.857436 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:37.857445 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:37.857788 (+ 343us) client_negotiation.cc:769] Sending connection context
0927 07:00:37.858069 (+ 281us) client_negotiation.cc:240] Negotiation successful
0927 07:00:37.858600 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":266,"threads_started":1}
I20260927 07:00:37.861356 17705 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39751
I20260927 07:00:37.861646 17705 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:37.862365 17705 heartbeater.cc:511] Master 127.13.121.254:39751 requested a full tablet report, sending...
I20260927 07:00:37.863896 17382 ts_manager.cc:194] Registered new tserver with Master: 4377952f654643b5807b6c28da59fa20 (127.13.121.194:38957)
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:00:37.864722 17382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:51415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:00:37.889899 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39751
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:38.209501 17710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:38.209801 17710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:38.209908 17710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:38.217392 17710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:38.217523 17710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:38.217670 17710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:38.227674 17710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:38.228960 17710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:38.230243 17710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:38.243505 17716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:38.243569 17718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:38.243534 17715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:38.244244 17710 server_base.cc:1034] running on GCE node
I20260927 07:00:38.244940 17710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:38.245961 17710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:38.247202 17710 hybrid_clock.cc:648] HybridClock initialized: now 1790492438247110 us; error 55 us; skew 500 ppm
I20260927 07:00:38.247529 17710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492438, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:38.254518 17710 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:00:38.256605 17710 webserver.cc:460] Webserver started at http://127.13.121.195:44533/ using document root <none> and password file <none>
I20260927 07:00:38.257354 17710 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:38.257464 17710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:38.257761 17710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:38.261399 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "b1146f654e24416c93d32b0206c00fbf"
format_stamp: "Formatted at 2026-09-27 07:00:38 on dist-test-slave-p6hz"
server_key: "1824cffe9cc548e9ee1f7431c52c14f7"
server_key_iv: "b166f148e908b4b47f02d84abb0bf17c"
server_key_version: "encryptionkey@0"
I20260927 07:00:38.262594 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "b1146f654e24416c93d32b0206c00fbf"
format_stamp: "Formatted at 2026-09-27 07:00:38 on dist-test-slave-p6hz"
server_key: "1824cffe9cc548e9ee1f7431c52c14f7"
server_key_iv: "b166f148e908b4b47f02d84abb0bf17c"
server_key_version: "encryptionkey@0"
I20260927 07:00:38.268361 17710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260927 07:00:38.272176 17725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:38.273581 17710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:00:38.273718 17710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "b1146f654e24416c93d32b0206c00fbf"
format_stamp: "Formatted at 2026-09-27 07:00:38 on dist-test-slave-p6hz"
server_key: "1824cffe9cc548e9ee1f7431c52c14f7"
server_key_iv: "b166f148e908b4b47f02d84abb0bf17c"
server_key_version: "encryptionkey@0"
I20260927 07:00:38.273901 17710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:38.294798 17710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:38.299182 17710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:38.299451 17710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:38.300350 17710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:38.301649 17710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:38.301718 17710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:38.301808 17710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:38.301849 17710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:38.329938 17710 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:42189
I20260927 07:00:38.329984 17837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:42189 every 8 connection(s)
I20260927 07:00:38.331166 17710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:38.338069 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 17710
I20260927 07:00:38.338248 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:38.338629 13799 external_mini_cluster.cc:1428] Setting key 320ee5d4b6ef62c3c4355e1bef063edd
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492438, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:38.351056 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:38.333669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:60731 (local address 127.13.121.254:39751)
0927 07:00:38.333840 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:38.333844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:38.334976 (+ 1132us) server_negotiation.cc:407] Connection header received
0927 07:00:38.336737 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:38.336743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:38.336821 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:38.336903 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:38.338396 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.339178 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.340978 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.341408 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.341506 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:38.345299 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:38.345305 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:38.345308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:38.345341 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:38.347908 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.348829 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.348833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.348835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.348967 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.349395 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.349400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.349402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.349684 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:38.349853 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:38.350577 (+ 724us) server_negotiation.cc:299] Negotiation successful
0927 07:00:38.350769 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 07:00:38.351958 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:38.334150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39751 (local address 127.13.121.195:60731)
0927 07:00:38.334821 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:38.334862 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:38.336482 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:38.337151 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:38.337159 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:38.337668 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:38.338217 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:38.338240 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.339324 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.339328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:38.340841 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:38.340856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.341645 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.341654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:38.341934 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:38.342562 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:38.342588 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:38.345062 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:38.348063 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:38.348073 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:38.348089 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:38.348659 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:38.349083 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:38.349087 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:38.349089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:38.349270 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:38.349858 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:38.349864 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:38.350196 (+ 332us) client_negotiation.cc:769] Sending connection context
0927 07:00:38.350438 (+ 242us) client_negotiation.cc:240] Negotiation successful
0927 07:00:38.350836 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":250,"threads_started":1}
I20260927 07:00:38.353353 17838 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39751
I20260927 07:00:38.353683 17838 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:38.354518 17838 heartbeater.cc:511] Master 127.13.121.254:39751 requested a full tablet report, sending...
I20260927 07:00:38.355877 17381 ts_manager.cc:194] Registered new tserver with Master: b1146f654e24416c93d32b0206c00fbf (127.13.121.195:42189)
I20260927 07:00:38.356762 17381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:60731
I20260927 07:00:38.366966 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:00:38.378873 17571 heartbeater.cc:503] Master 127.13.121.254:39751 was elected leader, sending a full tablet report...
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:38.412967 17845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:38.397929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.13.121.193:40145)
0927 07:00:38.398505 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:38.398514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:38.398546 (+ 32us) server_negotiation.cc:407] Connection header received
0927 07:00:38.398688 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:38.398695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:38.398915 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:38.399051 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:38.400450 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.401474 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.402750 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.403381 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.403497 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:38.407270 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:38.407284 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:38.407344 (+ 60us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:38.407395 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:38.410137 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.410817 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.410824 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.410827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.410949 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.411393 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.411398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.411400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.411747 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:38.412029 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:38.412336 (+ 307us) server_negotiation.cc:299] Negotiation successful
0927 07:00:38.412611 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":248,"threads_started":1}
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:38.430310 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:38.417100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52210 (local address 127.13.121.254:39751)
0927 07:00:38.417304 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:38.417311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:38.417466 (+ 155us) server_negotiation.cc:407] Connection header received
0927 07:00:38.417642 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:38.417648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:38.417733 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:38.417825 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:38.419070 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.419858 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.421096 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.421489 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.421599 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:38.425282 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:38.425288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:38.425291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:38.425329 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:38.427811 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.428462 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.428467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.428469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.428595 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.428936 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.428940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.428942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.429260 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:38.429485 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:38.429816 (+ 331us) server_negotiation.cc:299] Negotiation successful
0927 07:00:38.430024 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260927 07:00:38.430902 17381 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52210
I20260927 07:00:38.431959 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17443
I20260927 07:00:38.444983 17845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:38.432899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56594 (local address 127.13.121.193:40145)
0927 07:00:38.433099 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:38.433106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:38.433264 (+ 158us) server_negotiation.cc:407] Connection header received
0927 07:00:38.433431 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:38.433435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:38.433527 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:38.433611 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:38.434946 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.435670 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.436866 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.437272 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.437382 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:38.439700 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:38.439705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:38.439708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:38.439745 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:38.442378 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.443118 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.443122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.443125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.443293 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.443646 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.443651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.443654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.443918 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:38.444137 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:38.444455 (+ 318us) server_negotiation.cc:299] Negotiation successful
0927 07:00:38.444666 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:00:38.445639 17566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:38.528867 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 17443
I20260927 07:00:38.542237 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17577
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:00:38.559175 17850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:38.543162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53942 (local address 127.13.121.194:38957)
0927 07:00:38.543650 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:38.543658 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:38.543685 (+ 27us) server_negotiation.cc:407] Connection header received
0927 07:00:38.543757 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:38.543763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:38.543970 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:38.544107 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:38.545213 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.546201 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.547557 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.548173 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.548305 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:38.552677 (+ 4372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:38.552694 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:38.552712 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:38.552759 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:38.555976 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.556815 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.556824 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.556826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.556969 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.557400 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.557404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.557406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.557784 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:38.558017 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:38.558446 (+ 429us) server_negotiation.cc:299] Negotiation successful
0927 07:00:38.558737 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":260,"threads_started":1}
I20260927 07:00:38.560734 17700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:38.642429 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 17577
I20260927 07:00:38.655373 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17710
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:00:38.673532 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:38.656395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.13.121.195:42189)
0927 07:00:38.657303 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:38.657313 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:38.657358 (+ 45us) server_negotiation.cc:407] Connection header received
0927 07:00:38.657446 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:38.657455 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:38.657748 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:38.657980 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:38.659983 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.661110 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.662514 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:38.663238 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:38.663363 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:38.667729 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:38.667744 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:38.667762 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:38.667811 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:38.670588 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.671386 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.671393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.671395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.671537 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:38.671916 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:38.671920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:38.671923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:38.672288 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:38.672597 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:38.672906 (+ 309us) server_negotiation.cc:299] Negotiation successful
0927 07:00:38.673186 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":415,"threads_started":1}
I20260927 07:00:38.674903 17833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:38.757475 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 17710
I20260927 07:00:38.770381 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17350
I20260927 07:00:38.771593 17412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:38.872956 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 17350
2026-09-27T07:00:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3761 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17857](info): set up 2 sockets
Sep 27 07:00:38 dist-test-slave-p6hz krb5kdc[17857](info): commencing operation
krb5kdc: starting...
W20260927 07:00:39.875540 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.965s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:39 dist-test-slave-p6hz krb5kdc[17857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:40.062978 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:35495
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:35495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:40.398379 17870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:40.398707 17870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:40.398837 17870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:40.406265 17870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:40.406416 17870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:40.406497 17870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:40.406570 17870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:40.406635 17870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:40.416824 17870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:35495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:35495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:40.418179 17870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:40.419585 17870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:40.430953 17875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:40.430917 17878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:40.430994 17876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:40.432085 17870 server_base.cc:1034] running on GCE node
I20260927 07:00:40.432890 17870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:40.434369 17870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:40.435612 17870 hybrid_clock.cc:648] HybridClock initialized: now 1790492440435555 us; error 50 us; skew 500 ppm
I20260927 07:00:40.435995 17870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:40 dist-test-slave-p6hz krb5kdc[17857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492440, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:40.439056 17870 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:40.440276 17870 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:40.440382 17870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:40.440704 17870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:40.443934 17870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "feb0dbe1ba0f4084a25ace01a0d29d22"
format_stamp: "Formatted at 2026-09-27 07:00:40 on dist-test-slave-p6hz"
server_key: "e7fb976bf3917cce70cdc8beeea785e1"
server_key_iv: "c2a4cd6548b6b48f1da5d7640b88d149"
server_key_version: "encryptionkey@0"
I20260927 07:00:40.444751 17870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "feb0dbe1ba0f4084a25ace01a0d29d22"
format_stamp: "Formatted at 2026-09-27 07:00:40 on dist-test-slave-p6hz"
server_key: "e7fb976bf3917cce70cdc8beeea785e1"
server_key_iv: "c2a4cd6548b6b48f1da5d7640b88d149"
server_key_version: "encryptionkey@0"
I20260927 07:00:40.450132 17870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:00:40.454130 17884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:40.455689 17870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:00:40.455920 17870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "feb0dbe1ba0f4084a25ace01a0d29d22"
format_stamp: "Formatted at 2026-09-27 07:00:40 on dist-test-slave-p6hz"
server_key: "e7fb976bf3917cce70cdc8beeea785e1"
server_key_iv: "c2a4cd6548b6b48f1da5d7640b88d149"
server_key_version: "encryptionkey@0"
I20260927 07:00:40.456161 17870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:40.481187 17870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:40.485357 17870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:40.485599 17870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:40.502473 17870 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:35495
I20260927 07:00:40.502480 17935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:35495 every 8 connection(s)
I20260927 07:00:40.503417 17870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:40.506970 17936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:40.511639 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 17870
I20260927 07:00:40.511854 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:40.512614 13799 external_mini_cluster.cc:1428] Setting key cdd1bd41d9bb56e45ae7e294c48dafcb
I20260927 07:00:40.518687 17936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22: Bootstrap starting.
I20260927 07:00:40.521945 17936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:40.523028 17936 log.cc:824] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:40.525875 17936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22: No bootstrap required, opened a new log
Sep 27 07:00:40 dist-test-slave-p6hz krb5kdc[17857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:40.535079 17936 raft_consensus.cc:348] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb0dbe1ba0f4084a25ace01a0d29d22" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35495 } }
I20260927 07:00:40.535511 17936 raft_consensus.cc:374] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:40.535625 17936 raft_consensus.cc:729] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feb0dbe1ba0f4084a25ace01a0d29d22, State: Initialized, Role: FOLLOWER
I20260927 07:00:40.535681 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:40.514761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.13.121.254:35495)
0927 07:00:40.515589 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:40.515602 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:40.515643 (+ 41us) server_negotiation.cc:407] Connection header received
0927 07:00:40.517558 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:40.517585 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:40.518026 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:40.518424 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:40.519910 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:40.521257 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:40.522580 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:40.523367 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:40.523539 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:40.527741 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:40.527750 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:40.527770 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:40.527825 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:40.531112 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:40.532093 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:40.532102 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:40.532109 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:40.532323 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:40.532726 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:40.532730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:40.532732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:40.533396 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:40.533614 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:40.534108 (+ 494us) server_negotiation.cc:299] Negotiation successful
0927 07:00:40.534541 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":371,"threads_started":1}
I20260927 07:00:40.536232 17936 consensus_queue.cc:260] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb0dbe1ba0f4084a25ace01a0d29d22" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35495 } }
I20260927 07:00:40.536482 17936 raft_consensus.cc:388] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:40.536604 17936 raft_consensus.cc:482] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:40.536770 17936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:40.539677 17936 raft_consensus.cc:504] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb0dbe1ba0f4084a25ace01a0d29d22" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35495 } }
I20260927 07:00:40.540216 17936 leader_election.cc:302] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feb0dbe1ba0f4084a25ace01a0d29d22; no voters:
I20260927 07:00:40.540908 17936 leader_election.cc:288] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:40.540995 17941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:40.541673 17941 raft_consensus.cc:686] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [term 1 LEADER]: Becoming Leader. State: Replica: feb0dbe1ba0f4084a25ace01a0d29d22, State: Running, Role: LEADER
I20260927 07:00:40.542197 17941 consensus_queue.cc:237] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb0dbe1ba0f4084a25ace01a0d29d22" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35495 } }
I20260927 07:00:40.543000 17936 sys_catalog.cc:564] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:40.545122 17942 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "feb0dbe1ba0f4084a25ace01a0d29d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb0dbe1ba0f4084a25ace01a0d29d22" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35495 } } }
I20260927 07:00:40.545516 17942 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:40.545935 17943 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader feb0dbe1ba0f4084a25ace01a0d29d22. Latest consensus state: current_term: 1 leader_uuid: "feb0dbe1ba0f4084a25ace01a0d29d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb0dbe1ba0f4084a25ace01a0d29d22" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35495 } } }
I20260927 07:00:40.546289 17943 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:40.548849 17948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:40.553910 17948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:40.563195 17948 catalog_manager.cc:1347] Generated new cluster ID: e8b4378b1e2e48cbbda5595ca91d1d7d
I20260927 07:00:40.563337 17948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:40.596305 17948 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:40.597707 17948 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:40.609375 17948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22: Generated new TSK 0
I20260927 07:00:40.610298 17948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:00:40.671033 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35495
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:41.004233 17962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:41.004525 17962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:41.004621 17962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:41.011848 17962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:41.011962 17962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:41.012079 17962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:41.021723 17962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:41.023002 17962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:41.024307 17962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:41.037016 17967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:41.036893 17970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:41.036890 17968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:41.037479 17962 server_base.cc:1034] running on GCE node
I20260927 07:00:41.038321 17962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:41.039484 17962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:41.040702 17962 hybrid_clock.cc:648] HybridClock initialized: now 1790492441040649 us; error 41 us; skew 500 ppm
I20260927 07:00:41.041047 17962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:41 dist-test-slave-p6hz krb5kdc[17857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492441, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:41.044273 17962 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:00:41.045362 17962 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:41.045455 17962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:41.045717 17962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:41.048969 17962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "8fd2e1944d17417e871102dcd14e4778"
format_stamp: "Formatted at 2026-09-27 07:00:41 on dist-test-slave-p6hz"
server_key: "d4d7bf114bb157254f52eb0be1c93a2f"
server_key_iv: "e23be016c2c60f3eff85b9bb8a55eea8"
server_key_version: "encryptionkey@0"
I20260927 07:00:41.049757 17962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "8fd2e1944d17417e871102dcd14e4778"
format_stamp: "Formatted at 2026-09-27 07:00:41 on dist-test-slave-p6hz"
server_key: "d4d7bf114bb157254f52eb0be1c93a2f"
server_key_iv: "e23be016c2c60f3eff85b9bb8a55eea8"
server_key_version: "encryptionkey@0"
I20260927 07:00:41.055095 17962 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:00:41.058929 17976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:41.060613 17962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:00:41.060811 17962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "8fd2e1944d17417e871102dcd14e4778"
format_stamp: "Formatted at 2026-09-27 07:00:41 on dist-test-slave-p6hz"
server_key: "d4d7bf114bb157254f52eb0be1c93a2f"
server_key_iv: "e23be016c2c60f3eff85b9bb8a55eea8"
server_key_version: "encryptionkey@0"
I20260927 07:00:41.061039 17962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:41.084895 17962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:41.089306 17962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:41.089586 17962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:41.090549 17962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:41.091998 17962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:41.092090 17962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:41.092170 17962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:41.092222 17962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:41.124166 17962 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:40957
I20260927 07:00:41.124220 18088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:40957 every 8 connection(s)
I20260927 07:00:41.124993 17962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:41.131920 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 17962
I20260927 07:00:41.132123 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:41.132721 13799 external_mini_cluster.cc:1428] Setting key fefd953b619b7d0f6578c121cbe31005
Sep 27 07:00:41 dist-test-slave-p6hz krb5kdc[17857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492441, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:41.146412 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:41.127439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:42983 (local address 127.13.121.254:35495)
0927 07:00:41.127876 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:41.127881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:41.128880 (+ 999us) server_negotiation.cc:407] Connection header received
0927 07:00:41.130737 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:41.130743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:41.130849 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:41.130944 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:41.132681 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:41.133530 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:41.135188 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:41.135603 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:41.135717 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:41.139706 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:41.139712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:41.139715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:41.139754 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:41.142401 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:41.143600 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:41.143606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:41.143609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:41.143832 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:41.144430 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:41.144437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:41.144441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:41.144757 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:41.144885 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:41.145920 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0927 07:00:41.146103 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":228,"threads_started":1}
I20260927 07:00:41.147996 18092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:41.127919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:35495 (local address 127.13.121.193:42983)
0927 07:00:41.128658 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:41.128699 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:41.130431 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:41.131235 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:41.131245 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:41.131838 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:41.132490 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:41.132511 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:41.133694 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:41.133698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:41.134998 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:41.135012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:41.135851 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:41.135860 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:41.136084 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:41.136817 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:41.136844 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:41.139420 (+ 2576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:41.142600 (+ 3180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:41.142611 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:41.142632 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:41.143385 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:41.143973 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:41.143978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:41.143981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:41.144236 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:41.144896 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:41.144905 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:41.145411 (+ 506us) client_negotiation.cc:769] Sending connection context
0927 07:00:41.145736 (+ 325us) client_negotiation.cc:240] Negotiation successful
0927 07:00:41.146326 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":314,"threads_started":1}
I20260927 07:00:41.149644 18089 heartbeater.cc:347] Connected to a master server at 127.13.121.254:35495
I20260927 07:00:41.149972 18089 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:00:41.150656 18089 heartbeater.cc:511] Master 127.13.121.254:35495 requested a full tablet report, sending...
I20260927 07:00:41.152863 17901 ts_manager.cc:194] Registered new tserver with Master: 8fd2e1944d17417e871102dcd14e4778 (127.13.121.193:40957)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
I20260927 07:00:41.154721 17901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:42983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:00:41.178573 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35495
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:41.505820 18095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:41.506121 18095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:41.506251 18095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:41.513639 18095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:41.513777 18095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:41.513933 18095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:41.524226 18095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:41.525632 18095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:41.527006 18095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:41.539793 18100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:41.539853 18103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:41.539821 18101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:41.540549 18095 server_base.cc:1034] running on GCE node
I20260927 07:00:41.541162 18095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:41.542120 18095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:41.543330 18095 hybrid_clock.cc:648] HybridClock initialized: now 1790492441543264 us; error 49 us; skew 500 ppm
I20260927 07:00:41.543641 18095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:41 dist-test-slave-p6hz krb5kdc[17857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492441, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:41.546718 18095 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:00:41.547945 18095 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:41.548046 18095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:41.548352 18095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:41.551366 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "218d3eed784b48849ba400c6627db016"
format_stamp: "Formatted at 2026-09-27 07:00:41 on dist-test-slave-p6hz"
server_key: "ca7f16a795517f9202faa65020e626fb"
server_key_iv: "5f96956bb0e205f422ca00d25ad6028a"
server_key_version: "encryptionkey@0"
I20260927 07:00:41.552145 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "218d3eed784b48849ba400c6627db016"
format_stamp: "Formatted at 2026-09-27 07:00:41 on dist-test-slave-p6hz"
server_key: "ca7f16a795517f9202faa65020e626fb"
server_key_iv: "5f96956bb0e205f422ca00d25ad6028a"
server_key_version: "encryptionkey@0"
I20260927 07:00:41.557120 18095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:00:41.560642 18109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:41.562047 18095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 07:00:41.562193 18095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "218d3eed784b48849ba400c6627db016"
format_stamp: "Formatted at 2026-09-27 07:00:41 on dist-test-slave-p6hz"
server_key: "ca7f16a795517f9202faa65020e626fb"
server_key_iv: "5f96956bb0e205f422ca00d25ad6028a"
server_key_version: "encryptionkey@0"
I20260927 07:00:41.562402 18095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:41.591472 18095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:41.595760 18095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:41.596030 18095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:41.597020 18095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:41.598392 18095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:41.598488 18095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:41.598590 18095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:41.598656 18095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:41.628237 18095 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:37783
I20260927 07:00:41.628293 18221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:37783 every 8 connection(s)
I20260927 07:00:41.629034 18095 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:41.637668 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 18095
I20260927 07:00:41.637831 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:41.638257 13799 external_mini_cluster.cc:1428] Setting key e0553c8dbf7b55b828d08c7a0acc0cd1
Sep 27 07:00:41 dist-test-slave-p6hz krb5kdc[17857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492441, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:41.649299 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:41.631252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:57515 (local address 127.13.121.254:35495)
0927 07:00:41.631446 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:41.631450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:41.632625 (+ 1175us) server_negotiation.cc:407] Connection header received
0927 07:00:41.634362 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:41.634376 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:41.634456 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:41.634539 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:41.636083 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:41.636774 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:41.638335 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:41.638788 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:41.638898 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:41.642947 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:41.642953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:41.642958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:41.642997 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:41.646071 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:41.646851 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:41.646855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:41.646858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:41.647002 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:41.647466 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:41.647470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:41.647472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:41.647755 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:41.647905 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:41.648712 (+ 807us) server_negotiation.cc:299] Negotiation successful
0927 07:00:41.648946 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 07:00:41.650331 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:41.631753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:35495 (local address 127.13.121.194:57515)
0927 07:00:41.632469 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:41.632516 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:41.634108 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:41.634805 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:41.634814 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:41.635360 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:41.635909 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:41.635926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:41.636927 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:41.636931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:41.638204 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:41.638216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:41.639010 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:41.639018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:41.639304 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:41.639991 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:41.640018 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:41.642666 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:41.646234 (+ 3568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:41.646245 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:41.646274 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:41.646712 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:41.647116 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:41.647121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:41.647124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:41.647347 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:41.647891 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:41.647899 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:41.648353 (+ 454us) client_negotiation.cc:769] Sending connection context
0927 07:00:41.648616 (+ 263us) client_negotiation.cc:240] Negotiation successful
0927 07:00:41.649164 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":293,"threads_started":1}
I20260927 07:00:41.651960 18222 heartbeater.cc:347] Connected to a master server at 127.13.121.254:35495
I20260927 07:00:41.652263 18222 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:41.652967 18222 heartbeater.cc:511] Master 127.13.121.254:35495 requested a full tablet report, sending...
I20260927 07:00:41.654507 17901 ts_manager.cc:194] Registered new tserver with Master: 218d3eed784b48849ba400c6627db016 (127.13.121.194:37783)
I20260927 07:00:41.655352 17901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:57515
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:00:41.682762 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35495
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:42.005575 18227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:42.005911 18227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:42.006037 18227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:42.013973 18227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:42.014125 18227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:42.014284 18227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:42.024495 18227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:42.025840 18227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:42.027232 18227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:42.040279 18232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:42.040253 18235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:42.040304 18233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:42.041124 18227 server_base.cc:1034] running on GCE node
I20260927 07:00:42.041781 18227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:42.042774 18227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:42.043973 18227 hybrid_clock.cc:648] HybridClock initialized: now 1790492442043921 us; error 43 us; skew 500 ppm
I20260927 07:00:42.044337 18227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:42 dist-test-slave-p6hz krb5kdc[17857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492442, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:42.047844 18227 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:00:42.049088 18227 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:42.049192 18227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:42.049515 18227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:42.052769 18227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "4739c6252e1b4cc3bc2362360bcbb402"
format_stamp: "Formatted at 2026-09-27 07:00:42 on dist-test-slave-p6hz"
server_key: "0e6482268f54346a65269888dde37302"
server_key_iv: "542a6bc04ce09eb584acc179709ea92e"
server_key_version: "encryptionkey@0"
I20260927 07:00:42.053625 18227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "4739c6252e1b4cc3bc2362360bcbb402"
format_stamp: "Formatted at 2026-09-27 07:00:42 on dist-test-slave-p6hz"
server_key: "0e6482268f54346a65269888dde37302"
server_key_iv: "542a6bc04ce09eb584acc179709ea92e"
server_key_version: "encryptionkey@0"
I20260927 07:00:42.059196 18227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:00:42.063545 18241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:42.065218 18227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260927 07:00:42.065385 18227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "4739c6252e1b4cc3bc2362360bcbb402"
format_stamp: "Formatted at 2026-09-27 07:00:42 on dist-test-slave-p6hz"
server_key: "0e6482268f54346a65269888dde37302"
server_key_iv: "542a6bc04ce09eb584acc179709ea92e"
server_key_version: "encryptionkey@0"
I20260927 07:00:42.065611 18227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:42.090961 18227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:42.095202 18227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:42.095448 18227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:42.096490 18227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:42.099467 18227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:42.099586 18227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:42.099695 18227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:42.099814 18227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260927 07:00:42.131352 18227 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:37931
I20260927 07:00:42.131422 18353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:37931 every 8 connection(s)
I20260927 07:00:42.132154 18227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:42.141196 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 18227
I20260927 07:00:42.141497 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:42.141964 13799 external_mini_cluster.cc:1428] Setting key 244ea80ca57e1e404f0cb2a2f7c95928
Sep 27 07:00:42 dist-test-slave-p6hz krb5kdc[17857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492442, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:42.154093 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.134517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:46269 (local address 127.13.121.254:35495)
0927 07:00:42.134723 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.134728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.136042 (+ 1314us) server_negotiation.cc:407] Connection header received
0927 07:00:42.137834 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.137841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.137948 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.138040 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:42.139832 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.140665 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.142528 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.143075 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.143223 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.147950 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:42.147955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:42.147959 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:42.147996 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:42.150633 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:42.151625 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:42.151631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:42.151633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:42.151779 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:42.152313 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:42.152318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:42.152320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:42.152637 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:42.152771 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.153596 (+ 825us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.153825 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260927 07:00:42.155026 18356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.135053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:35495 (local address 127.13.121.195:46269)
0927 07:00:42.135869 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:42.135917 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:42.137556 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:42.138248 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:42.138257 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:42.138776 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:42.139604 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.139624 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.140894 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.140899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.142341 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.142356 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.143350 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.143356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.143682 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.144579 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:42.144606 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:42.147673 (+ 3067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:42.150846 (+ 3173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:42.150857 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:42.150881 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:42.151475 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:42.151937 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:42.151941 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:42.151944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:42.152156 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:42.152790 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:42.152797 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:42.153185 (+ 388us) client_negotiation.cc:769] Sending connection context
0927 07:00:42.153448 (+ 263us) client_negotiation.cc:240] Negotiation successful
0927 07:00:42.153878 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":339,"threads_started":1}
I20260927 07:00:42.156703 18354 heartbeater.cc:347] Connected to a master server at 127.13.121.254:35495
I20260927 07:00:42.157025 18354 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:42.157742 18354 heartbeater.cc:511] Master 127.13.121.254:35495 requested a full tablet report, sending...
I20260927 07:00:42.158152 18089 heartbeater.cc:503] Master 127.13.121.254:35495 was elected leader, sending a full tablet report...
I20260927 07:00:42.159490 17900 ts_manager.cc:194] Registered new tserver with Master: 4739c6252e1b4cc3bc2362360bcbb402 (127.13.121.195:37931)
I20260927 07:00:42.160331 17900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:46269
I20260927 07:00:42.171036 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:00:42.188257 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.175777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60536 (local address 127.13.121.254:35495)
0927 07:00:42.175998 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.176005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.176381 (+ 376us) server_negotiation.cc:407] Connection header received
0927 07:00:42.176513 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.176517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.176584 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.176670 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:42.178088 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.178820 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.180247 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.180685 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.180785 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.183107 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:42.183112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:42.183115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:42.183181 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:42.185682 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:42.186394 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:42.186398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:42.186401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:42.186528 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:42.186896 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:42.186900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:42.186902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:42.187316 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:42.187507 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.187778 (+ 271us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.187954 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:00:42.192601 17900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:00:42.199393 17900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:42.234855 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.212497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47720 (local address 127.13.121.193:40957)
0927 07:00:42.223306 (+ 10809us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.223317 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.224941 (+ 1624us) server_negotiation.cc:407] Connection header received
0927 07:00:42.225294 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.225303 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.225564 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.225752 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:42.226842 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.228497 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.231777 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.233252 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.233441 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.233633 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.234355 (+ 722us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.234532 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10541,"thread_start_us":10839,"threads_started":1}
I20260927 07:00:42.235857 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.211694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:37931 (local address 127.0.0.1:34270)
0927 07:00:42.224750 (+ 13056us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:42.224796 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:42.225027 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:42.226028 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:42.226037 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:42.226084 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:42.226641 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.226659 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.228659 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.228664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.231580 (+ 2916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.231597 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.234853 (+ 3256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.234859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.235026 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.235056 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 07:00:42.235128 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 07:00:42.235363 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12774,"mutex_wait_us":207,"spinlock_wait_cycles":4608,"thread_start_us":12526,"threads_started":1}
I20260927 07:00:42.236487 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.211923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.13.121.195:37931)
0927 07:00:42.224073 (+ 12150us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.224082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.225222 (+ 1140us) server_negotiation.cc:407] Connection header received
0927 07:00:42.225311 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.225317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.225563 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.225752 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:42.226859 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.228489 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.231783 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.233632 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.233809 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.233973 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.236081 (+ 2108us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.236236 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11966,"thread_start_us":11796,"threads_started":1}
I20260927 07:00:42.235890 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.212132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:40957 (local address 127.0.0.1:47720)
0927 07:00:42.224750 (+ 12618us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:42.224796 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:42.225054 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:42.226029 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:42.226037 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:42.226076 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:42.226661 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.226669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.228663 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.228667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.231580 (+ 2913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.231597 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.233854 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.233860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.234066 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.234127 (+ 61us) client_negotiation.cc:769] Sending connection context
0927 07:00:42.235360 (+ 1233us) client_negotiation.cc:240] Negotiation successful
0927 07:00:42.235480 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12422,"mutex_wait_us":263,"spinlock_wait_cycles":1792,"thread_start_us":12290,"threads_started":1}
I20260927 07:00:42.241073 18289 tablet_service.cc:1467] Processing CreateTablet for tablet 870eb6a154ed46a4ae030555922f8539 (DEFAULT_TABLE table=test-table [id=f72577602f824c1fbdd72493af39ba26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:42.241492 18024 tablet_service.cc:1467] Processing CreateTablet for tablet 870eb6a154ed46a4ae030555922f8539 (DEFAULT_TABLE table=test-table [id=f72577602f824c1fbdd72493af39ba26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:42.242651 18289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 870eb6a154ed46a4ae030555922f8539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:42.242772 18024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 870eb6a154ed46a4ae030555922f8539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:42.246462 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.225710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:37783 (local address 127.0.0.1:57648)
0927 07:00:42.236973 (+ 11263us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:42.236992 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:42.237099 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:42.238281 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:42.238286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:42.238314 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:42.238478 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.238489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.240558 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.240564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.242382 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.242400 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.245229 (+ 2829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.245236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.245816 (+ 580us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.245855 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 07:00:42.245963 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 07:00:42.246129 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11157,"thread_start_us":11875,"threads_started":1}
I20260927 07:00:42.247244 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.225947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57648 (local address 127.13.121.194:37783)
0927 07:00:42.237486 (+ 11539us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.237495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.237523 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:00:42.237615 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.237623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.237886 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.238073 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:42.238653 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.240374 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.242570 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.244027 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.244847 (+ 820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.245105 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.246729 (+ 1624us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.246947 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11372,"thread_start_us":11263,"threads_started":1}
I20260927 07:00:42.251468 18157 tablet_service.cc:1467] Processing CreateTablet for tablet 870eb6a154ed46a4ae030555922f8539 (DEFAULT_TABLE table=test-table [id=f72577602f824c1fbdd72493af39ba26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:42.253481 18157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 870eb6a154ed46a4ae030555922f8539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:42.255615 18372 tablet_bootstrap.cc:492] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: Bootstrap starting.
I20260927 07:00:42.259244 18372 tablet_bootstrap.cc:654] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:42.259960 18373 tablet_bootstrap.cc:492] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: Bootstrap starting.
I20260927 07:00:42.260804 18372 log.cc:824] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:42.263618 18373 tablet_bootstrap.cc:654] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:42.264192 18372 tablet_bootstrap.cc:492] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: No bootstrap required, opened a new log
I20260927 07:00:42.264521 18372 ts_tablet_manager.cc:1397] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260927 07:00:42.264937 18373 log.cc:824] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:42.269934 18373 tablet_bootstrap.cc:492] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: No bootstrap required, opened a new log
I20260927 07:00:42.270219 18373 ts_tablet_manager.cc:1397] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260927 07:00:42.271703 18376 tablet_bootstrap.cc:492] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: Bootstrap starting.
I20260927 07:00:42.274583 18376 tablet_bootstrap.cc:654] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:42.274644 18372 raft_consensus.cc:348] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } }
I20260927 07:00:42.275187 18372 raft_consensus.cc:374] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:42.275293 18372 raft_consensus.cc:729] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd2e1944d17417e871102dcd14e4778, State: Initialized, Role: FOLLOWER
I20260927 07:00:42.275741 18376 log.cc:824] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:42.275943 18372 consensus_queue.cc:260] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } }
I20260927 07:00:42.278748 18376 tablet_bootstrap.cc:492] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: No bootstrap required, opened a new log
I20260927 07:00:42.279070 18376 ts_tablet_manager.cc:1397] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260927 07:00:42.281185 18373 raft_consensus.cc:348] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } }
I20260927 07:00:42.281730 18373 raft_consensus.cc:374] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:42.281833 18373 raft_consensus.cc:729] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4739c6252e1b4cc3bc2362360bcbb402, State: Initialized, Role: FOLLOWER
I20260927 07:00:42.282583 18373 consensus_queue.cc:260] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } }
I20260927 07:00:42.283392 18372 ts_tablet_manager.cc:1428] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: Time spent starting tablet: real 0.019s user 0.007s sys 0.012s
I20260927 07:00:42.284332 18354 heartbeater.cc:503] Master 127.13.121.254:35495 was elected leader, sending a full tablet report...
I20260927 07:00:42.286170 18373 ts_tablet_manager.cc:1428] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260927 07:00:42.289165 18376 raft_consensus.cc:348] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } }
I20260927 07:00:42.289553 18376 raft_consensus.cc:374] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:42.289647 18376 raft_consensus.cc:729] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 218d3eed784b48849ba400c6627db016, State: Initialized, Role: FOLLOWER
I20260927 07:00:42.290292 18376 consensus_queue.cc:260] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } }
I20260927 07:00:42.291483 18222 heartbeater.cc:503] Master 127.13.121.254:35495 was elected leader, sending a full tablet report...
I20260927 07:00:42.292006 18376 ts_tablet_manager.cc:1428] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
W20260927 07:00:42.377208 18090 tablet.cc:2369] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:00:42.380820 18223 tablet.cc:2369] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:00:42.383522 18355 tablet.cc:2369] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:00:42.653259 18378 raft_consensus.cc:482] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:42.653563 18378 raft_consensus.cc:504] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } }
I20260927 07:00:42.655318 18378 leader_election.cc:288] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4739c6252e1b4cc3bc2362360bcbb402 (127.13.121.195:37931), 218d3eed784b48849ba400c6627db016 (127.13.121.194:37783)
I20260927 07:00:42.663534 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.655756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:55889 (local address 127.13.121.195:37931)
0927 07:00:42.655949 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.655955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.656821 (+ 866us) server_negotiation.cc:407] Connection header received
0927 07:00:42.656939 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.656944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.657004 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.657071 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:42.657615 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.658642 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.661226 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.662489 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.662626 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.662702 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.663112 (+ 410us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.663271 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 07:00:42.663534 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.656086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:51655 (local address 127.13.121.194:37783)
0927 07:00:42.656281 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.656296 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.656362 (+ 66us) server_negotiation.cc:407] Connection header received
0927 07:00:42.656667 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.656672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.656731 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.656822 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:42.657402 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.658442 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.661223 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.662453 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.662626 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.662702 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.663129 (+ 427us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.663271 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 07:00:42.663763 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.655581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:37783 (local address 127.13.121.193:51655)
0927 07:00:42.656250 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:42.656271 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:42.656490 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:42.656858 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:42.656862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:42.656884 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:42.657250 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.657263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.658577 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.658581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.661026 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.661036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.662706 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.662713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.662988 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.663013 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 07:00:42.663078 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 07:00:42.663328 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":129,"thread_start_us":556,"threads_started":1}
I20260927 07:00:42.664139 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.655731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:37931 (local address 127.13.121.193:55889)
0927 07:00:42.656722 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:42.656735 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:42.656833 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:42.657121 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:42.657126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:42.657146 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:42.657471 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.657478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.658773 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.658777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.661019 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:42.661036 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.662724 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.662728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:42.662915 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.662975 (+ 60us) client_negotiation.cc:769] Sending connection context
0927 07:00:42.663075 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 07:00:42.663339 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":89,"thread_start_us":656,"threads_started":1}
I20260927 07:00:42.664745 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870eb6a154ed46a4ae030555922f8539" candidate_uuid: "8fd2e1944d17417e871102dcd14e4778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "218d3eed784b48849ba400c6627db016" is_pre_election: true
I20260927 07:00:42.664966 18309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870eb6a154ed46a4ae030555922f8539" candidate_uuid: "8fd2e1944d17417e871102dcd14e4778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4739c6252e1b4cc3bc2362360bcbb402" is_pre_election: true
I20260927 07:00:42.665221 18177 raft_consensus.cc:2454] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd2e1944d17417e871102dcd14e4778 in term 0.
I20260927 07:00:42.665411 18309 raft_consensus.cc:2454] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd2e1944d17417e871102dcd14e4778 in term 0.
I20260927 07:00:42.665966 17980 leader_election.cc:302] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 218d3eed784b48849ba400c6627db016, 8fd2e1944d17417e871102dcd14e4778; no voters:
I20260927 07:00:42.666424 18378 raft_consensus.cc:2781] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:42.666566 18378 raft_consensus.cc:482] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:42.666661 18378 raft_consensus.cc:3037] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:42.668888 18378 raft_consensus.cc:504] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } }
I20260927 07:00:42.669523 18378 leader_election.cc:288] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [CANDIDATE]: Term 1 election: Requested vote from peers 4739c6252e1b4cc3bc2362360bcbb402 (127.13.121.195:37931), 218d3eed784b48849ba400c6627db016 (127.13.121.194:37783)
I20260927 07:00:42.669994 18309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870eb6a154ed46a4ae030555922f8539" candidate_uuid: "8fd2e1944d17417e871102dcd14e4778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4739c6252e1b4cc3bc2362360bcbb402"
I20260927 07:00:42.669994 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "870eb6a154ed46a4ae030555922f8539" candidate_uuid: "8fd2e1944d17417e871102dcd14e4778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "218d3eed784b48849ba400c6627db016"
I20260927 07:00:42.670213 18309 raft_consensus.cc:3037] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:42.670203 18177 raft_consensus.cc:3037] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:42.673266 18309 raft_consensus.cc:2454] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd2e1944d17417e871102dcd14e4778 in term 1.
I20260927 07:00:42.673269 18177 raft_consensus.cc:2454] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd2e1944d17417e871102dcd14e4778 in term 1.
I20260927 07:00:42.673710 17978 leader_election.cc:302] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4739c6252e1b4cc3bc2362360bcbb402, 8fd2e1944d17417e871102dcd14e4778; no voters:
I20260927 07:00:42.674093 18378 raft_consensus.cc:2781] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:42.674502 18378 raft_consensus.cc:686] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd2e1944d17417e871102dcd14e4778, State: Running, Role: LEADER
I20260927 07:00:42.674973 18378 consensus_queue.cc:237] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } }
I20260927 07:00:42.679745 17900 catalog_manager.cc:5495] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd2e1944d17417e871102dcd14e4778 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "8fd2e1944d17417e871102dcd14e4778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fd2e1944d17417e871102dcd14e4778" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 40957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 } health_report { overall_health: UNKNOWN } } }
I20260927 07:00:42.705821 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.700272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47722 (local address 127.13.121.193:40957)
0927 07:00:42.700484 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.700490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.700690 (+ 200us) server_negotiation.cc:407] Connection header received
0927 07:00:42.700942 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.700946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.701026 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.701164 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:00:42.701603 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.702283 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.703717 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.704134 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.704245 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.704717 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:00:42.705238 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:00:42.705392 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.705483 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.705568 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:00:42.714200 18177 raft_consensus.cc:1261] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [term 1 FOLLOWER]: Refusing update from remote peer 8fd2e1944d17417e871102dcd14e4778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:42.714201 18309 raft_consensus.cc:1261] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [term 1 FOLLOWER]: Refusing update from remote peer 8fd2e1944d17417e871102dcd14e4778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:42.715369 18383 consensus_queue.cc:1035] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4739c6252e1b4cc3bc2362360bcbb402" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:42.715756 18378 consensus_queue.cc:1035] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "218d3eed784b48849ba400c6627db016" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:42.741621 17900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:42.741856 17900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:42.745288 17900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P feb0dbe1ba0f4084a25ace01a0d29d22: Sending DeleteTablet for 3 replicas of tablet 870eb6a154ed46a4ae030555922f8539
I20260927 07:00:42.746484 18289 tablet_service.cc:1514] Processing DeleteTablet for tablet 870eb6a154ed46a4ae030555922f8539 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:42 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:34270
I20260927 07:00:42.746465 18024 tablet_service.cc:1514] Processing DeleteTablet for tablet 870eb6a154ed46a4ae030555922f8539 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:42 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:47720
I20260927 07:00:42.746694 18157 tablet_service.cc:1514] Processing DeleteTablet for tablet 870eb6a154ed46a4ae030555922f8539 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:42 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:57648
I20260927 07:00:42.747399 18397 tablet_replica.cc:330] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: stopping tablet replica
I20260927 07:00:42.747793 18398 tablet_replica.cc:330] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: stopping tablet replica
I20260927 07:00:42.748067 18397 raft_consensus.cc:2229] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:42.748180 18398 raft_consensus.cc:2229] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:00:42.748731 18397 raft_consensus.cc:2258] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:42.748773 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17962
I20260927 07:00:42.748915 18398 raft_consensus.cc:2258] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:42.751160 18397 ts_tablet_manager.cc:1905] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:42.751184 18398 ts_tablet_manager.cc:1905] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:42.759650 18399 tablet_replica.cc:330] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: stopping tablet replica
I20260927 07:00:42.760157 18397 ts_tablet_manager.cc:1918] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:00:42.760196 18398 ts_tablet_manager.cc:1918] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:00:42.760206 18399 raft_consensus.cc:2229] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:42.760392 18398 log.cc:1196] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/870eb6a154ed46a4ae030555922f8539
I20260927 07:00:42.760589 18397 log.cc:1196] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/870eb6a154ed46a4ae030555922f8539
I20260927 07:00:42.760773 18399 raft_consensus.cc:2258] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:42.760998 18398 ts_tablet_manager.cc:1939] T 870eb6a154ed46a4ae030555922f8539 P 8fd2e1944d17417e871102dcd14e4778: Deleting consensus metadata
I20260927 07:00:42.761318 18397 ts_tablet_manager.cc:1939] T 870eb6a154ed46a4ae030555922f8539 P 4739c6252e1b4cc3bc2362360bcbb402: Deleting consensus metadata
I20260927 07:00:42.762938 17888 catalog_manager.cc:4841] TS 8fd2e1944d17417e871102dcd14e4778 (127.13.121.193:40957): tablet 870eb6a154ed46a4ae030555922f8539 (table test-table [id=f72577602f824c1fbdd72493af39ba26]) successfully deleted
I20260927 07:00:42.763192 18399 ts_tablet_manager.cc:1905] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:42.763492 17886 catalog_manager.cc:4841] TS 4739c6252e1b4cc3bc2362360bcbb402 (127.13.121.195:37931): tablet 870eb6a154ed46a4ae030555922f8539 (table test-table [id=f72577602f824c1fbdd72493af39ba26]) successfully deleted
Sep 27 07:00:42 dist-test-slave-p6hz krb5kdc[17857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:42.768362 18399 ts_tablet_manager.cc:1918] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:00:42.768520 18399 log.cc:1196] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/870eb6a154ed46a4ae030555922f8539
I20260927 07:00:42.768913 18399 ts_tablet_manager.cc:1939] T 870eb6a154ed46a4ae030555922f8539 P 218d3eed784b48849ba400c6627db016: Deleting consensus metadata
I20260927 07:00:42.770524 17888 catalog_manager.cc:4841] TS 218d3eed784b48849ba400c6627db016 (127.13.121.194:37783): tablet 870eb6a154ed46a4ae030555922f8539 (table test-table [id=f72577602f824c1fbdd72493af39ba26]) successfully deleted
I20260927 07:00:42.774632 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.750198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.13.121.193:40957)
0927 07:00:42.750364 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.750370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.757947 (+ 7577us) server_negotiation.cc:407] Connection header received
0927 07:00:42.758271 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.758276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.758370 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.758475 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:42.760375 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.761603 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.763535 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.764067 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.764341 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.768348 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:42.768361 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:42.768377 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:42.768421 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:42.771826 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:42.772505 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:42.772515 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:42.772519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:42.772705 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:42.773068 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:42.773073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:42.773077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:42.773543 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:42.773836 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.773996 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.774235 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:00:42.775406 18084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:42.870364 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 17962
I20260927 07:00:42.884250 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 18095
Sep 27 07:00:42 dist-test-slave-p6hz krb5kdc[17857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:00:42.898394 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:42.884972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.13.121.194:37783)
0927 07:00:42.885148 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:42.885152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:42.885219 (+ 67us) server_negotiation.cc:407] Connection header received
0927 07:00:42.885385 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:42.885389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:42.885449 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:42.885522 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:42.886605 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.887380 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.888625 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:42.889029 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:42.889129 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:42.893104 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:42.893121 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:42.893138 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:42.893192 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:42.895938 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:42.896596 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:42.896603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:42.896606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:42.896727 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:42.897039 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:42.897042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:42.897044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:42.897421 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:42.897622 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:42.897884 (+ 262us) server_negotiation.cc:299] Negotiation successful
0927 07:00:42.898044 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 07:00:42.899039 18217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:42.986848 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 18095
I20260927 07:00:42.999893 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 18227
Sep 27 07:00:43 dist-test-slave-p6hz krb5kdc[17857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:00:43.014228 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:43.000713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34280 (local address 127.13.121.195:37931)
0927 07:00:43.000929 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:43.000934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:43.000955 (+ 21us) server_negotiation.cc:407] Connection header received
0927 07:00:43.001019 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:43.001023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:43.001086 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:43.001175 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:43.002457 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:43.003233 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:43.004455 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:43.004869 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:43.004963 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:43.009039 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:43.009054 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:43.009070 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:43.009114 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:43.011876 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:43.012569 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:43.012576 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:43.012579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:43.012704 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:43.013027 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:43.013030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:43.013032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:43.013367 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:43.013574 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:43.013809 (+ 235us) server_negotiation.cc:299] Negotiation successful
0927 07:00:43.013971 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 07:00:43.014899 18349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:43.104719 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 18227
I20260927 07:00:43.118221 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 17870
I20260927 07:00:43.119379 17931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:43.224514 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 17870
2026-09-27T07:00:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4352 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-27T07:00:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:43Z Disabled control of system clock
I20260927 07:00:43.285454 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:43287
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:43287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 07:00:43.606474 18408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:43.606777 18408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:43.606896 18408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:43.614218 18408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:43.614327 18408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:43.614364 18408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:43.614437 18408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:43.614491 18408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:43.623817 18408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:43287
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:43287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:43.625103 18408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:43.626948 18408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:43.637837 18413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:43.637759 18414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:43.637814 18416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:43.638620 18408 server_base.cc:1034] running on GCE node
I20260927 07:00:43.639417 18408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:43.640813 18408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:43.642019 18408 hybrid_clock.cc:648] HybridClock initialized: now 1790492443641963 us; error 44 us; skew 500 ppm
I20260927 07:00:43.642346 18408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:00:43.648872 18408 webserver.cc:460] Webserver started at http://127.13.121.254:38977/ using document root <none> and password file <none>
I20260927 07:00:43.649609 18408 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:43.649716 18408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:43.650028 18408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:43.653192 18408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "3d7bb25fb18e45ce984e375ebad5a8b5"
format_stamp: "Formatted at 2026-09-27 07:00:43 on dist-test-slave-p6hz"
server_key: "5b64fcbd82704bae9dcc986eef188d59"
server_key_iv: "2f6a7982199835df5b04ea1ea0a9423a"
server_key_version: "encryptionkey@0"
I20260927 07:00:43.654052 18408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "3d7bb25fb18e45ce984e375ebad5a8b5"
format_stamp: "Formatted at 2026-09-27 07:00:43 on dist-test-slave-p6hz"
server_key: "5b64fcbd82704bae9dcc986eef188d59"
server_key_iv: "2f6a7982199835df5b04ea1ea0a9423a"
server_key_version: "encryptionkey@0"
I20260927 07:00:43.659332 18408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260927 07:00:43.662947 18422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:43.664399 18408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:00:43.664570 18408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "3d7bb25fb18e45ce984e375ebad5a8b5"
format_stamp: "Formatted at 2026-09-27 07:00:43 on dist-test-slave-p6hz"
server_key: "5b64fcbd82704bae9dcc986eef188d59"
server_key_iv: "2f6a7982199835df5b04ea1ea0a9423a"
server_key_version: "encryptionkey@0"
I20260927 07:00:43.664811 18408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:43.670439 18408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:43.671231 18408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:43.671489 18408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:43.688441 18408 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:43287
I20260927 07:00:43.688460 18473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:43287 every 8 connection(s)
I20260927 07:00:43.689725 18408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:43.693110 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 18408
I20260927 07:00:43.693279 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:43.693621 18474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:43.693913 13799 external_mini_cluster.cc:1428] Setting key 714ed697a85a6184b7e6b244c532a773
I20260927 07:00:43.703183 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:43.695990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49608 (local address 127.13.121.254:43287)
0927 07:00:43.697125 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:43.697149 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:43.697201 (+ 52us) server_negotiation.cc:407] Connection header received
0927 07:00:43.698716 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:43.698744 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:43.699369 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:43.699857 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:43.701036 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:43.701049 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:43.701072 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:43.701090 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:43.701170 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:43.701191 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:43.701447 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:43.701589 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 07:00:43.701889 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":361,"threads_started":1}
I20260927 07:00:43.708331 18474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5: Bootstrap starting.
I20260927 07:00:43.711519 18474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:43.712520 18474 log.cc:824] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:43.715535 18474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5: No bootstrap required, opened a new log
I20260927 07:00:43.723079 18474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7bb25fb18e45ce984e375ebad5a8b5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43287 } }
I20260927 07:00:43.723546 18474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:43.723619 18474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7bb25fb18e45ce984e375ebad5a8b5, State: Initialized, Role: FOLLOWER
I20260927 07:00:43.724164 18474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7bb25fb18e45ce984e375ebad5a8b5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43287 } }
I20260927 07:00:43.724367 18474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:43.724463 18474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:43.724589 18474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:43.726722 18474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7bb25fb18e45ce984e375ebad5a8b5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43287 } }
I20260927 07:00:43.727331 18474 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7bb25fb18e45ce984e375ebad5a8b5; no voters:
I20260927 07:00:43.728021 18474 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:43.728133 18479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:43.728767 18479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [term 1 LEADER]: Becoming Leader. State: Replica: 3d7bb25fb18e45ce984e375ebad5a8b5, State: Running, Role: LEADER
I20260927 07:00:43.729368 18479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7bb25fb18e45ce984e375ebad5a8b5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43287 } }
I20260927 07:00:43.729987 18474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:43.732158 18480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d7bb25fb18e45ce984e375ebad5a8b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7bb25fb18e45ce984e375ebad5a8b5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43287 } } }
I20260927 07:00:43.732151 18481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7bb25fb18e45ce984e375ebad5a8b5. Latest consensus state: current_term: 1 leader_uuid: "3d7bb25fb18e45ce984e375ebad5a8b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7bb25fb18e45ce984e375ebad5a8b5" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43287 } } }
I20260927 07:00:43.732569 18480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:43.732574 18481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:43.733976 18486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:43.738575 18486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:43.746228 18486 catalog_manager.cc:1347] Generated new cluster ID: c4e8725ab7f84dbb8870982de6334dbd
I20260927 07:00:43.746359 18486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:43.760434 18486 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:43.760913 18486 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:43.771191 18486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5: Generated new TSK 0
I20260927 07:00:43.772162 18486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:00:43.787552 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43287
--builtin_ntp_servers=127.13.121.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 07:00:44.109127 18498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:44.109438 18498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:44.109549 18498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:44.116968 18498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:44.117079 18498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:44.117223 18498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:44.126708 18498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:44.128016 18498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:44.129832 18498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:44.143090 18503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:44.143205 18504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:44.143213 18506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:44.144176 18498 server_base.cc:1034] running on GCE node
I20260927 07:00:44.144809 18498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:44.145766 18498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:44.146972 18498 hybrid_clock.cc:648] HybridClock initialized: now 1790492444146925 us; error 45 us; skew 500 ppm
I20260927 07:00:44.147379 18498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:00:44.149848 18498 webserver.cc:460] Webserver started at http://127.13.121.193:36101/ using document root <none> and password file <none>
I20260927 07:00:44.150636 18498 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:44.150734 18498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:44.151059 18498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:44.154066 18498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "5519fddf7bac4582976def1ae0f5677e"
format_stamp: "Formatted at 2026-09-27 07:00:44 on dist-test-slave-p6hz"
server_key: "55781f6d19f575214ce0961da5cce3bf"
server_key_iv: "e27a53e7aaefb335c00909b52fc81a84"
server_key_version: "encryptionkey@0"
I20260927 07:00:44.154968 18498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "5519fddf7bac4582976def1ae0f5677e"
format_stamp: "Formatted at 2026-09-27 07:00:44 on dist-test-slave-p6hz"
server_key: "55781f6d19f575214ce0961da5cce3bf"
server_key_iv: "e27a53e7aaefb335c00909b52fc81a84"
server_key_version: "encryptionkey@0"
I20260927 07:00:44.160831 18498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:00:44.164601 18512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:44.166272 18498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:44.166458 18498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "5519fddf7bac4582976def1ae0f5677e"
format_stamp: "Formatted at 2026-09-27 07:00:44 on dist-test-slave-p6hz"
server_key: "55781f6d19f575214ce0961da5cce3bf"
server_key_iv: "e27a53e7aaefb335c00909b52fc81a84"
server_key_version: "encryptionkey@0"
I20260927 07:00:44.166667 18498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:44.172590 18498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:44.173396 18498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:44.173636 18498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:44.174643 18498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:44.176141 18498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:44.176226 18498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:44.176323 18498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:44.176388 18498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:44.206365 18498 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:42211
I20260927 07:00:44.206420 18624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:42211 every 8 connection(s)
I20260927 07:00:44.207705 18498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:44.215696 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 18498
I20260927 07:00:44.215915 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:44.216065 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:44.210213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:60435 (local address 127.13.121.254:43287)
0927 07:00:44.210612 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:44.210616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:44.211983 (+ 1367us) server_negotiation.cc:407] Connection header received
0927 07:00:44.213767 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:44.213772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:44.213868 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:44.213990 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:44.215338 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:44.215344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:44.215349 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:44.215370 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:44.215409 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:44.215421 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:44.215528 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:44.215704 (+ 176us) server_negotiation.cc:299] Negotiation successful
0927 07:00:44.215771 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":194,"threads_started":1}
I20260927 07:00:44.216646 13799 external_mini_cluster.cc:1428] Setting key 7f52354733df5f0b66cabc378fe6c995
I20260927 07:00:44.217079 18628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:44.210816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:43287 (local address 127.13.121.193:60435)
0927 07:00:44.211813 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:44.211858 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:44.213465 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:44.214227 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:44.214236 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:44.214342 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:44.214997 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:00:44.214999 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:44.215040 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:00:44.215164 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:44.215505 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:44.215515 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:44.215522 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 07:00:44.215780 (+ 258us) client_negotiation.cc:240] Negotiation successful
0927 07:00:44.216071 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":306,"threads_started":1}
I20260927 07:00:44.218519 18625 heartbeater.cc:347] Connected to a master server at 127.13.121.254:43287
I20260927 07:00:44.218860 18625 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:44.219545 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43287
--builtin_ntp_servers=127.13.121.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260927 07:00:44.219647 18625 heartbeater.cc:511] Master 127.13.121.254:43287 requested a full tablet report, sending...
I20260927 07:00:44.221944 18439 ts_manager.cc:194] Registered new tserver with Master: 5519fddf7bac4582976def1ae0f5677e (127.13.121.193:42211)
W20260927 07:00:44.539113 18629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:44.539469 18629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:44.539561 18629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:44.547066 18629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:44.547205 18629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:44.547325 18629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:44.557484 18629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:44.558794 18629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:44.560669 18629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:44.574251 18635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:44.574203 18637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:44.574294 18634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:44.575505 18629 server_base.cc:1034] running on GCE node
I20260927 07:00:44.576154 18629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:44.577207 18629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:44.578438 18629 hybrid_clock.cc:648] HybridClock initialized: now 1790492444578367 us; error 66 us; skew 500 ppm
I20260927 07:00:44.578804 18629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:00:44.582538 18629 webserver.cc:460] Webserver started at http://127.13.121.194:41199/ using document root <none> and password file <none>
I20260927 07:00:44.583360 18629 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:44.583475 18629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:44.583770 18629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:44.586742 18629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "5c49d5b75de64ec99af5e866580a91e1"
format_stamp: "Formatted at 2026-09-27 07:00:44 on dist-test-slave-p6hz"
server_key: "a4a854673076d074f56be1a0d23625a0"
server_key_iv: "6aee2d3484c3d1735e83a6491fd216fb"
server_key_version: "encryptionkey@0"
I20260927 07:00:44.587803 18629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "5c49d5b75de64ec99af5e866580a91e1"
format_stamp: "Formatted at 2026-09-27 07:00:44 on dist-test-slave-p6hz"
server_key: "a4a854673076d074f56be1a0d23625a0"
server_key_iv: "6aee2d3484c3d1735e83a6491fd216fb"
server_key_version: "encryptionkey@0"
I20260927 07:00:44.593518 18629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:00:44.597698 18643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:44.599279 18629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:00:44.599466 18629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "5c49d5b75de64ec99af5e866580a91e1"
format_stamp: "Formatted at 2026-09-27 07:00:44 on dist-test-slave-p6hz"
server_key: "a4a854673076d074f56be1a0d23625a0"
server_key_iv: "6aee2d3484c3d1735e83a6491fd216fb"
server_key_version: "encryptionkey@0"
I20260927 07:00:44.599686 18629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:44.605501 18629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:44.606317 18629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:44.606567 18629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:44.607520 18629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:44.608956 18629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:44.609046 18629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:44.609151 18629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:44.609216 18629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:44.637257 18629 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:40323
I20260927 07:00:44.637310 18755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:40323 every 8 connection(s)
I20260927 07:00:44.638598 18629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:44.646661 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:44.641173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:45317 (local address 127.13.121.254:43287)
0927 07:00:44.641370 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:44.641376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:44.642927 (+ 1551us) server_negotiation.cc:407] Connection header received
0927 07:00:44.644576 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:44.644580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:44.644637 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:44.644729 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:44.646009 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:44.646014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:44.646017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:44.646035 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:44.646066 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:44.646078 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:44.646178 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:44.646384 (+ 206us) server_negotiation.cc:299] Negotiation successful
0927 07:00:44.646444 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 07:00:44.647598 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:44.641742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:43287 (local address 127.13.121.194:45317)
0927 07:00:44.642727 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:44.642778 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:44.644299 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:44.644958 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:44.644967 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:44.645064 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:44.645685 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:00:44.645687 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:44.645730 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:00:44.645817 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:44.646159 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:44.646165 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:44.646170 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:00:44.646372 (+ 202us) client_negotiation.cc:240] Negotiation successful
0927 07:00:44.646583 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":367,"threads_started":1}
I20260927 07:00:44.648622 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 18629
I20260927 07:00:44.648782 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:44.649096 18756 heartbeater.cc:347] Connected to a master server at 127.13.121.254:43287
I20260927 07:00:44.649418 18756 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:44.649565 13799 external_mini_cluster.cc:1428] Setting key 8e827e4d1a5cfa5edf41cb8af81c0f8a
I20260927 07:00:44.650044 18756 heartbeater.cc:511] Master 127.13.121.254:43287 requested a full tablet report, sending...
I20260927 07:00:44.651645 18438 ts_manager.cc:194] Registered new tserver with Master: 5c49d5b75de64ec99af5e866580a91e1 (127.13.121.194:40323)
I20260927 07:00:44.652849 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43287
--builtin_ntp_servers=127.13.121.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 07:00:44.977921 18759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:44.978209 18759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:44.978320 18759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:44.985574 18759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:44.985683 18759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:44.985837 18759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:44.994964 18759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:44.996330 18759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:44.998102 18759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:45.011215 18764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:45.011191 18767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:45.011610 18765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:45.012565 18759 server_base.cc:1034] running on GCE node
I20260927 07:00:45.013202 18759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:45.014183 18759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:45.015375 18759 hybrid_clock.cc:648] HybridClock initialized: now 1790492445015308 us; error 58 us; skew 500 ppm
I20260927 07:00:45.015746 18759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:00:45.018204 18759 webserver.cc:460] Webserver started at http://127.13.121.195:46811/ using document root <none> and password file <none>
I20260927 07:00:45.018872 18759 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:45.018955 18759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:45.019279 18759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:45.022099 18759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "9355557074304f3d8c7c9254e50eef7f"
format_stamp: "Formatted at 2026-09-27 07:00:45 on dist-test-slave-p6hz"
server_key: "e91f94d38765b5842147c00e8ea4920d"
server_key_iv: "7837de517a5c243111381e86345738e5"
server_key_version: "encryptionkey@0"
I20260927 07:00:45.022907 18759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "9355557074304f3d8c7c9254e50eef7f"
format_stamp: "Formatted at 2026-09-27 07:00:45 on dist-test-slave-p6hz"
server_key: "e91f94d38765b5842147c00e8ea4920d"
server_key_iv: "7837de517a5c243111381e86345738e5"
server_key_version: "encryptionkey@0"
I20260927 07:00:45.028146 18759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260927 07:00:45.031791 18773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:45.033354 18759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:45.033533 18759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "9355557074304f3d8c7c9254e50eef7f"
format_stamp: "Formatted at 2026-09-27 07:00:45 on dist-test-slave-p6hz"
server_key: "e91f94d38765b5842147c00e8ea4920d"
server_key_iv: "7837de517a5c243111381e86345738e5"
server_key_version: "encryptionkey@0"
I20260927 07:00:45.033749 18759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:45.039505 18759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:45.040320 18759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:45.040565 18759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:45.041512 18759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:45.042938 18759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:45.043025 18759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:45.043128 18759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:45.043226 18759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260927 07:00:45.074018 18759 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:46169
I20260927 07:00:45.074064 18885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:46169 every 8 connection(s)
I20260927 07:00:45.075341 18759 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:45.081018 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 18759
I20260927 07:00:45.081182 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:45.081700 13799 external_mini_cluster.cc:1428] Setting key c335bef9ad4f9fae0b6dea24a48eb827
I20260927 07:00:45.084208 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.077830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:54809 (local address 127.13.121.254:43287)
0927 07:00:45.078031 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.078037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.079570 (+ 1533us) server_negotiation.cc:407] Connection header received
0927 07:00:45.081629 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.081633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.081689 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.081836 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.083399 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.083404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.083407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.083425 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.083461 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.083473 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.083608 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.083831 (+ 223us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.083892 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 07:00:45.085232 18888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.078390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:43287 (local address 127.13.121.195:54809)
0927 07:00:45.079377 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.079421 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:45.081263 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:45.082075 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:45.082089 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:45.082224 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:45.082936 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:00:45.082939 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:45.082995 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:00:45.083113 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:45.083581 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:45.083590 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:45.083596 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 07:00:45.083871 (+ 275us) client_negotiation.cc:240] Negotiation successful
0927 07:00:45.084134 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":261,"threads_started":1}
I20260927 07:00:45.086642 18886 heartbeater.cc:347] Connected to a master server at 127.13.121.254:43287
I20260927 07:00:45.086934 18886 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:45.087584 18886 heartbeater.cc:511] Master 127.13.121.254:43287 requested a full tablet report, sending...
I20260927 07:00:45.088987 18438 ts_manager.cc:194] Registered new tserver with Master: 9355557074304f3d8c7c9254e50eef7f (127.13.121.195:46169)
I20260927 07:00:45.097708 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:00:45.104796 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.102231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.13.121.254:43287)
0927 07:00:45.102446 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.102454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.102947 (+ 493us) server_negotiation.cc:407] Connection header received
0927 07:00:45.103104 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.103108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.103186 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.103274 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.104184 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.104190 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.104196 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.104214 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.104244 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.104256 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.104367 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.104461 (+ 94us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.104522 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260927 07:00:45.108915 18438 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:00:45.115523 18438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:45.147377 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.128022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.13.121.194:40323)
0927 07:00:45.143553 (+ 15531us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.143565 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.144101 (+ 536us) server_negotiation.cc:407] Connection header received
0927 07:00:45.144191 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.144202 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.144491 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.144645 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.146592 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.146602 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.146613 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.146628 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.146676 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.146694 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.146851 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.146946 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.147041 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15301,"thread_start_us":15562,"threads_started":1}
I20260927 07:00:45.147558 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.127793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:40323 (local address 127.0.0.1:49826)
0927 07:00:45.143647 (+ 15854us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.143670 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:45.143789 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:45.145684 (+ 1895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:45.145688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:45.145706 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:45.146312 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:00:45.146315 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:45.146347 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:00:45.146425 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:45.146777 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:45.146788 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:45.146797 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 07:00:45.146970 (+ 173us) client_negotiation.cc:240] Negotiation successful
0927 07:00:45.147058 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15686,"thread_start_us":11533,"threads_started":1}
I20260927 07:00:45.148048 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.127874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.13.121.193:42211)
0927 07:00:45.144717 (+ 16843us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.144726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.144749 (+ 23us) server_negotiation.cc:407] Connection header received
0927 07:00:45.144813 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.144820 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.145047 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.145866 (+ 819us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.146988 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.146996 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.147009 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.147031 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.147083 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.147102 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.147278 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.147668 (+ 390us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.147759 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16683,"thread_start_us":17564,"threads_started":1}
I20260927 07:00:45.148587 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.127748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42211 (local address 127.0.0.1:49638)
0927 07:00:45.139802 (+ 12054us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.139854 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:45.140057 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:45.145272 (+ 5215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:45.145281 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:45.145379 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:45.146760 (+ 1381us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:00:45.146763 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:45.146778 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:00:45.146816 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:45.147241 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:45.147246 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:45.147251 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:00:45.147420 (+ 169us) client_negotiation.cc:240] Negotiation successful
0927 07:00:45.147476 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11725,"spinlock_wait_cycles":9984,"thread_start_us":16117,"threads_started":1}
I20260927 07:00:45.149233 18903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.130267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54120 (local address 127.13.121.195:46169)
0927 07:00:45.144316 (+ 14049us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.144324 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.144713 (+ 389us) server_negotiation.cc:407] Connection header received
0927 07:00:45.144954 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.144963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.145114 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.145255 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.147692 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.147701 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.147712 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.147729 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.147782 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.148421 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.148593 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.148789 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.148879 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13884,"thread_start_us":13721,"threads_started":1}
I20260927 07:00:45.149996 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.129996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:46169 (local address 127.0.0.1:54120)
0927 07:00:45.144585 (+ 14589us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.144605 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:45.144802 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:45.146368 (+ 1566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:45.146373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:45.146407 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:45.147435 (+ 1028us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:00:45.147441 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:45.147457 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:00:45.147500 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:45.148645 (+ 1145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:45.148650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:45.148655 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:00:45.149534 (+ 879us) client_negotiation.cc:240] Negotiation successful
0927 07:00:45.149586 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14467,"thread_start_us":14306,"threads_started":1}
I20260927 07:00:45.151638 18691 tablet_service.cc:1467] Processing CreateTablet for tablet 892ec1b3a5d1422491965c3f28a7c929 (DEFAULT_TABLE table=test-table [id=b904cc4379784732abd91aac46aebd80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:45.152072 18560 tablet_service.cc:1467] Processing CreateTablet for tablet 892ec1b3a5d1422491965c3f28a7c929 (DEFAULT_TABLE table=test-table [id=b904cc4379784732abd91aac46aebd80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:45.153224 18560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 892ec1b3a5d1422491965c3f28a7c929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:45.153225 18691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 892ec1b3a5d1422491965c3f28a7c929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:45.153712 18821 tablet_service.cc:1467] Processing CreateTablet for tablet 892ec1b3a5d1422491965c3f28a7c929 (DEFAULT_TABLE table=test-table [id=b904cc4379784732abd91aac46aebd80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:00:45.155177 18821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 892ec1b3a5d1422491965c3f28a7c929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:45.165531 18904 tablet_bootstrap.cc:492] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e: Bootstrap starting.
I20260927 07:00:45.169179 18904 tablet_bootstrap.cc:654] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:45.170018 18906 tablet_bootstrap.cc:492] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: Bootstrap starting.
I20260927 07:00:45.170637 18904 log.cc:824] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:45.172580 18905 tablet_bootstrap.cc:492] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: Bootstrap starting.
I20260927 07:00:45.174201 18906 tablet_bootstrap.cc:654] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:45.175475 18906 log.cc:824] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:45.175558 18905 tablet_bootstrap.cc:654] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:45.176168 18904 tablet_bootstrap.cc:492] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e: No bootstrap required, opened a new log
I20260927 07:00:45.176501 18904 ts_tablet_manager.cc:1397] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20260927 07:00:45.176656 18905 log.cc:824] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:45.178763 18906 tablet_bootstrap.cc:492] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: No bootstrap required, opened a new log
I20260927 07:00:45.179092 18906 ts_tablet_manager.cc:1397] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260927 07:00:45.181404 18905 tablet_bootstrap.cc:492] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: No bootstrap required, opened a new log
I20260927 07:00:45.181638 18905 ts_tablet_manager.cc:1397] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260927 07:00:45.186623 18904 raft_consensus.cc:348] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } }
I20260927 07:00:45.187322 18904 raft_consensus.cc:374] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:45.187494 18904 raft_consensus.cc:729] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5519fddf7bac4582976def1ae0f5677e, State: Initialized, Role: FOLLOWER
I20260927 07:00:45.188366 18904 consensus_queue.cc:260] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } }
I20260927 07:00:45.189049 18905 raft_consensus.cc:348] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } }
I20260927 07:00:45.189581 18905 raft_consensus.cc:374] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:45.189603 18625 heartbeater.cc:503] Master 127.13.121.254:43287 was elected leader, sending a full tablet report...
I20260927 07:00:45.189724 18905 raft_consensus.cc:729] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c49d5b75de64ec99af5e866580a91e1, State: Initialized, Role: FOLLOWER
I20260927 07:00:45.189677 18906 raft_consensus.cc:348] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } }
I20260927 07:00:45.190147 18906 raft_consensus.cc:374] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:45.190284 18906 raft_consensus.cc:729] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9355557074304f3d8c7c9254e50eef7f, State: Initialized, Role: FOLLOWER
I20260927 07:00:45.190450 18904 ts_tablet_manager.cc:1428] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e: Time spent starting tablet: real 0.014s user 0.002s sys 0.011s
I20260927 07:00:45.190395 18905 consensus_queue.cc:260] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } }
I20260927 07:00:45.190981 18906 consensus_queue.cc:260] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } }
I20260927 07:00:45.191519 18756 heartbeater.cc:503] Master 127.13.121.254:43287 was elected leader, sending a full tablet report...
I20260927 07:00:45.192221 18905 ts_tablet_manager.cc:1428] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260927 07:00:45.192660 18911 raft_consensus.cc:482] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:45.192953 18911 raft_consensus.cc:504] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } }
I20260927 07:00:45.193707 18886 heartbeater.cc:503] Master 127.13.121.254:43287 was elected leader, sending a full tablet report...
I20260927 07:00:45.194397 18906 ts_tablet_manager.cc:1428] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260927 07:00:45.197862 18911 leader_election.cc:288] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5519fddf7bac4582976def1ae0f5677e (127.13.121.193:42211), 9355557074304f3d8c7c9254e50eef7f (127.13.121.195:46169)
I20260927 07:00:45.201052 18903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.195659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:58643 (local address 127.13.121.195:46169)
0927 07:00:45.195819 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.195826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.198623 (+ 2797us) server_negotiation.cc:407] Connection header received
0927 07:00:45.198869 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.198875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.198932 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.198998 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.200237 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.200244 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.200250 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.200273 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.200346 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.200365 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.200466 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.200743 (+ 277us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.200811 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:00:45.201229 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.195456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:54511 (local address 127.13.121.193:42211)
0927 07:00:45.196230 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.196236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.198612 (+ 2376us) server_negotiation.cc:407] Connection header received
0927 07:00:45.198869 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.198875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.198935 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.198998 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.200571 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.200577 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.200582 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.200601 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.200636 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.200651 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.200751 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.200952 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.201009 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654}
I20260927 07:00:45.201431 18914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.195277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:46169 (local address 127.13.121.194:58643)
0927 07:00:45.198545 (+ 3268us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.198560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:45.198726 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:45.199218 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:45.199222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:45.199249 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:45.200008 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:00:45.200012 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:45.200029 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:00:45.200077 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:45.200556 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:45.200560 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:45.200564 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 07:00:45.200761 (+ 197us) client_negotiation.cc:240] Negotiation successful
0927 07:00:45.200811 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3168,"spinlock_wait_cycles":20480,"thread_start_us":2301,"threads_started":1}
I20260927 07:00:45.201826 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.195063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42211 (local address 127.13.121.194:54511)
0927 07:00:45.198506 (+ 3443us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.198526 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:45.198726 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:45.199635 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:45.199640 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:45.199664 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:45.200329 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:00:45.200332 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:45.200399 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:00:45.200444 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:45.200797 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:45.200801 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:45.200806 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:00:45.201443 (+ 637us) client_negotiation.cc:240] Negotiation successful
0927 07:00:45.201483 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2679,"spinlock_wait_cycles":32128,"thread_start_us":3183,"threads_started":1}
I20260927 07:00:45.202284 18841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892ec1b3a5d1422491965c3f28a7c929" candidate_uuid: "5c49d5b75de64ec99af5e866580a91e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9355557074304f3d8c7c9254e50eef7f" is_pre_election: true
I20260927 07:00:45.202697 18580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892ec1b3a5d1422491965c3f28a7c929" candidate_uuid: "5c49d5b75de64ec99af5e866580a91e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5519fddf7bac4582976def1ae0f5677e" is_pre_election: true
I20260927 07:00:45.202867 18841 raft_consensus.cc:2454] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c49d5b75de64ec99af5e866580a91e1 in term 0.
I20260927 07:00:45.203238 18580 raft_consensus.cc:2454] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c49d5b75de64ec99af5e866580a91e1 in term 0.
I20260927 07:00:45.203589 18646 leader_election.cc:302] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c49d5b75de64ec99af5e866580a91e1, 9355557074304f3d8c7c9254e50eef7f; no voters:
I20260927 07:00:45.204015 18913 raft_consensus.cc:2781] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:00:45.204183 18913 raft_consensus.cc:482] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:00:45.204309 18913 raft_consensus.cc:3037] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:45.206809 18913 raft_consensus.cc:504] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } }
I20260927 07:00:45.207439 18913 leader_election.cc:288] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [CANDIDATE]: Term 1 election: Requested vote from peers 5519fddf7bac4582976def1ae0f5677e (127.13.121.193:42211), 9355557074304f3d8c7c9254e50eef7f (127.13.121.195:46169)
I20260927 07:00:45.207934 18580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892ec1b3a5d1422491965c3f28a7c929" candidate_uuid: "5c49d5b75de64ec99af5e866580a91e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5519fddf7bac4582976def1ae0f5677e"
I20260927 07:00:45.207932 18841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892ec1b3a5d1422491965c3f28a7c929" candidate_uuid: "5c49d5b75de64ec99af5e866580a91e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9355557074304f3d8c7c9254e50eef7f"
I20260927 07:00:45.208158 18580 raft_consensus.cc:3037] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:45.208146 18841 raft_consensus.cc:3037] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [term 0 FOLLOWER]: Advancing to term 1
W20260927 07:00:45.209870 18626 tablet.cc:2369] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:00:45.211584 18580 raft_consensus.cc:2454] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c49d5b75de64ec99af5e866580a91e1 in term 1.
I20260927 07:00:45.212111 18644 leader_election.cc:302] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5519fddf7bac4582976def1ae0f5677e, 5c49d5b75de64ec99af5e866580a91e1; no voters:
I20260927 07:00:45.212186 18841 raft_consensus.cc:2454] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c49d5b75de64ec99af5e866580a91e1 in term 1.
I20260927 07:00:45.212551 18913 raft_consensus.cc:2781] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:45.212875 18913 raft_consensus.cc:686] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 1 LEADER]: Becoming Leader. State: Replica: 5c49d5b75de64ec99af5e866580a91e1, State: Running, Role: LEADER
I20260927 07:00:45.213349 18913 consensus_queue.cc:237] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } }
I20260927 07:00:45.218525 18438 catalog_manager.cc:5495] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c49d5b75de64ec99af5e866580a91e1 (127.13.121.194). New cstate: current_term: 1 leader_uuid: "5c49d5b75de64ec99af5e866580a91e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c49d5b75de64ec99af5e866580a91e1" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 40323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 } health_report { overall_health: UNKNOWN } } }
I20260927 07:00:45.238417 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.236253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49838 (local address 127.13.121.194:40323)
0927 07:00:45.236442 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.236448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.236470 (+ 22us) server_negotiation.cc:407] Connection header received
0927 07:00:45.236732 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.236736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.236817 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.236905 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.237826 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.237831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.237834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.237849 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.237881 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.237892 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.238004 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.238120 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.238177 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:00:45.247115 18841 raft_consensus.cc:1261] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [term 1 FOLLOWER]: Refusing update from remote peer 5c49d5b75de64ec99af5e866580a91e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:45.247114 18580 raft_consensus.cc:1261] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [term 1 FOLLOWER]: Refusing update from remote peer 5c49d5b75de64ec99af5e866580a91e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:00:45.253752 18913 consensus_queue.cc:1035] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5519fddf7bac4582976def1ae0f5677e" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:45.254130 18911 consensus_queue.cc:1035] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9355557074304f3d8c7c9254e50eef7f" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 46169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:45.267236 18919 mvcc.cc:205] Tried to move back new op lower bound from 7333857055720751104 to 7333857055598010368. Current Snapshot: MvccSnapshot[applied={T|T < 7333857055720751104}]
I20260927 07:00:45.273675 18918 mvcc.cc:205] Tried to move back new op lower bound from 7333857055720751104 to 7333857055598010368. Current Snapshot: MvccSnapshot[applied={T|T < 7333857055720751104 or (T in {7333857055720751104})}]
I20260927 07:00:45.279158 18439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:45.279470 18439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:00:45.283046 18439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d7bb25fb18e45ce984e375ebad5a8b5: Sending DeleteTablet for 3 replicas of tablet 892ec1b3a5d1422491965c3f28a7c929
I20260927 07:00:45.284159 18560 tablet_service.cc:1514] Processing DeleteTablet for tablet 892ec1b3a5d1422491965c3f28a7c929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:45 UTC) from {username='slave'} at 127.0.0.1:49638
I20260927 07:00:45.284310 18821 tablet_service.cc:1514] Processing DeleteTablet for tablet 892ec1b3a5d1422491965c3f28a7c929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:45 UTC) from {username='slave'} at 127.0.0.1:54120
I20260927 07:00:45.285120 18691 tablet_service.cc:1514] Processing DeleteTablet for tablet 892ec1b3a5d1422491965c3f28a7c929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:00:45 UTC) from {username='slave'} at 127.0.0.1:49826
I20260927 07:00:45.285472 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 18498
I20260927 07:00:45.293650 18929 tablet_replica.cc:330] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: stopping tablet replica
I20260927 07:00:45.295022 18929 raft_consensus.cc:2229] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:00:45.296566 18929 raft_consensus.cc:2258] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:45.300830 18929 ts_tablet_manager.cc:1905] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:45.301785 18928 tablet_replica.cc:330] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: stopping tablet replica
I20260927 07:00:45.302410 18927 tablet_replica.cc:330] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e: stopping tablet replica
I20260927 07:00:45.302887 18928 raft_consensus.cc:2229] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:45.302878 18927 raft_consensus.cc:2229] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:45.303486 18927 raft_consensus.cc:2258] T 892ec1b3a5d1422491965c3f28a7c929 P 5519fddf7bac4582976def1ae0f5677e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:45.303486 18928 raft_consensus.cc:2258] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:45.304843 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.286940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49646 (local address 127.13.121.193:42211)
0927 07:00:45.287118 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.287123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.297613 (+ 10490us) server_negotiation.cc:407] Connection header received
0927 07:00:45.301176 (+ 3563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.301184 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.301421 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.301518 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.303982 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.303990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.303995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.304020 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.304061 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.304079 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.304189 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.304465 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.304561 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 07:00:45.305703 18928 ts_tablet_manager.cc:1905] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:45.305941 18620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:45.312600 18929 ts_tablet_manager.cc:1918] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:00:45.312927 18929 log.cc:1196] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/892ec1b3a5d1422491965c3f28a7c929
I20260927 07:00:45.313603 18929 ts_tablet_manager.cc:1939] T 892ec1b3a5d1422491965c3f28a7c929 P 5c49d5b75de64ec99af5e866580a91e1: Deleting consensus metadata
I20260927 07:00:45.313879 18928 ts_tablet_manager.cc:1918] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:00:45.314128 18928 log.cc:1196] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/892ec1b3a5d1422491965c3f28a7c929
I20260927 07:00:45.314700 18928 ts_tablet_manager.cc:1939] T 892ec1b3a5d1422491965c3f28a7c929 P 9355557074304f3d8c7c9254e50eef7f: Deleting consensus metadata
I20260927 07:00:45.315689 18424 catalog_manager.cc:4841] TS 5c49d5b75de64ec99af5e866580a91e1 (127.13.121.194:40323): tablet 892ec1b3a5d1422491965c3f28a7c929 (table test-table [id=b904cc4379784732abd91aac46aebd80]) successfully deleted
I20260927 07:00:45.316512 18425 catalog_manager.cc:4841] TS 9355557074304f3d8c7c9254e50eef7f (127.13.121.195:46169): tablet 892ec1b3a5d1422491965c3f28a7c929 (table test-table [id=b904cc4379784732abd91aac46aebd80]) successfully deleted
I20260927 07:00:45.400435 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 18498
W20260927 07:00:45.420437 18423 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.121.193:42211 (error 108)
I20260927 07:00:45.420674 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 18629
I20260927 07:00:45.422592 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.422103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42211 (local address 127.0.0.1:49658)
0927 07:00:45.422294 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.422430 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260927 07:00:45.422966 18423 catalog_manager.cc:4584] TS 5519fddf7bac4582976def1ae0f5677e (127.13.121.193:42211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 892ec1b3a5d1422491965c3f28a7c929: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42211: connect: Connection refused (error 111)
I20260927 07:00:45.424001 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.421846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49854 (local address 127.13.121.194:40323)
0927 07:00:45.422031 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.422037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.422055 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:00:45.422145 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.422150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.422213 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.422311 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.423264 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.423271 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.423276 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.423295 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.423333 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.423348 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.423504 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.423657 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.423714 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:00:45.424692 18751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:45.476826 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.476370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42211 (local address 127.0.0.1:49672)
0927 07:00:45.476584 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.476668 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260927 07:00:45.477862 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.477555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42211 (local address 127.0.0.1:49682)
0927 07:00:45.477684 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.477760 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 07:00:45.533599 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 18629
I20260927 07:00:45.538450 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.537858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42211 (local address 127.0.0.1:49688)
0927 07:00:45.538089 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.538237 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260927 07:00:45.539489 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.539104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42211 (local address 127.0.0.1:49702)
0927 07:00:45.539265 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.539348 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260927 07:00:45.548446 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 18759
I20260927 07:00:45.552713 18903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.549317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54136 (local address 127.13.121.195:46169)
0927 07:00:45.549477 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:45.549481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:45.549500 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:00:45.549741 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:45.549745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:45.549822 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:45.549929 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:45.551666 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:45.551673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:45.551678 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:45.551706 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:45.551763 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:45.551780 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:45.551975 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:45.552087 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 07:00:45.552159 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 07:00:45.553501 18881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:45.620060 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.619618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42211 (local address 127.0.0.1:49712)
0927 07:00:45.619809 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.619902 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260927 07:00:45.621155 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:45.620755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42211 (local address 127.0.0.1:49728)
0927 07:00:45.620964 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:45.621042 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260927 07:00:45.643448 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 18759
I20260927 07:00:45.656797 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 18408
I20260927 07:00:45.657900 18469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:45.762619 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 18408
2026-09-27T07:00:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2537 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260927 07:00:45.800120 13799 mini_oidc.cc:152] Starting JWKS server
I20260927 07:00:45.802685 13799 webserver.cc:460] Webserver started at https://127.0.0.1:34323/ using document root <none> and password file <none>
I20260927 07:00:45.803026 13799 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 07:00:45.803727 13799 webserver.cc:460] Webserver started at http://127.0.0.1:38495/ using document root <none> and password file <none>
2026-09-27T07:00:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:45Z Disabled control of system clock
I20260927 07:00:45.828258 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:45963
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:45963
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34323/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/testchainca.cert with env {}
W20260927 07:00:46.147002 18941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:46.147330 18941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:46.147419 18941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:46.154706 18941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:46.154826 18941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:46.154873 18941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:46.154928 18941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:46.154970 18941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 07:00:46.155030 18941 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 07:00:46.155068 18941 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34323/jwks.json
I20260927 07:00:46.165259 18941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:45963
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:45963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34323/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:46.166575 18941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:46.167857 18941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:46.179380 18946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:46.179445 18947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:46.179399 18949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:46.180302 18941 server_base.cc:1034] running on GCE node
I20260927 07:00:46.181135 18941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:46.182570 18941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:46.183776 18941 hybrid_clock.cc:648] HybridClock initialized: now 1790492446183721 us; error 46 us; skew 500 ppm
I20260927 07:00:46.184122 18941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:00:46.186445 18941 webserver.cc:460] Webserver started at http://127.13.121.254:46509/ using document root <none> and password file <none>
I20260927 07:00:46.187112 18941 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:46.187261 18941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:46.187554 18941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:46.190385 18941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "16b945f466c24e01bc710db136cde351"
format_stamp: "Formatted at 2026-09-27 07:00:46 on dist-test-slave-p6hz"
server_key: "6a8b57c4c5d7e721e6df817ca5e25224"
server_key_iv: "a25059ec4dba68edbfd8bb2f33f24756"
server_key_version: "encryptionkey@0"
I20260927 07:00:46.191201 18941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "16b945f466c24e01bc710db136cde351"
format_stamp: "Formatted at 2026-09-27 07:00:46 on dist-test-slave-p6hz"
server_key: "6a8b57c4c5d7e721e6df817ca5e25224"
server_key_iv: "a25059ec4dba68edbfd8bb2f33f24756"
server_key_version: "encryptionkey@0"
I20260927 07:00:46.196532 18941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:00:46.200338 18955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:46.201820 18941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:46.202005 18941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "16b945f466c24e01bc710db136cde351"
format_stamp: "Formatted at 2026-09-27 07:00:46 on dist-test-slave-p6hz"
server_key: "6a8b57c4c5d7e721e6df817ca5e25224"
server_key_iv: "a25059ec4dba68edbfd8bb2f33f24756"
server_key_version: "encryptionkey@0"
I20260927 07:00:46.202219 18941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:46.239794 18941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:46.240686 18941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:46.240996 18941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:46.258827 18941 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:45963
I20260927 07:00:46.258846 19006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:45963 every 8 connection(s)
I20260927 07:00:46.260206 18941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:46.264240 19007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:46.266763 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 18941
I20260927 07:00:46.266983 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:46.267704 13799 external_mini_cluster.cc:1428] Setting key 40a17deeeffdcd0bccf5ab568fc8780e
I20260927 07:00:46.276491 19007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351: Bootstrap starting.
I20260927 07:00:46.280069 19007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:46.281235 19007 log.cc:824] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:46.281800 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:46.269781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48682 (local address 127.13.121.254:45963)
0927 07:00:46.270824 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:46.270835 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:46.270879 (+ 44us) server_negotiation.cc:407] Connection header received
0927 07:00:46.272550 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:46.272574 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:46.273096 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:46.273450 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:46.274092 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:46.275494 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:46.277255 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:46.277931 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:46.278074 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:46.279336 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:46.279344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:46.279359 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:46.279921 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:46.280011 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:46.280032 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:46.280126 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:46.280374 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 07:00:46.280716 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":396,"threads_started":1}
I20260927 07:00:46.284364 19007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351: No bootstrap required, opened a new log
I20260927 07:00:46.285697 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:00:46.291946 19007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b945f466c24e01bc710db136cde351" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45963 } }
I20260927 07:00:46.292331 19007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:46.292430 19007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b945f466c24e01bc710db136cde351, State: Initialized, Role: FOLLOWER
I20260927 07:00:46.293025 19007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b945f466c24e01bc710db136cde351" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45963 } }
I20260927 07:00:46.293241 19007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:46.293370 19007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:46.293515 19007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:46.296108 19007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b945f466c24e01bc710db136cde351" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45963 } }
I20260927 07:00:46.296656 19007 leader_election.cc:302] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16b945f466c24e01bc710db136cde351; no voters:
I20260927 07:00:46.297308 19007 leader_election.cc:288] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:46.297446 19012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:46.298117 19012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [term 1 LEADER]: Becoming Leader. State: Replica: 16b945f466c24e01bc710db136cde351, State: Running, Role: LEADER
I20260927 07:00:46.298661 19012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b945f466c24e01bc710db136cde351" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45963 } }
I20260927 07:00:46.299479 19007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:46.301614 19013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16b945f466c24e01bc710db136cde351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b945f466c24e01bc710db136cde351" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45963 } } }
I20260927 07:00:46.302079 19013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:46.301716 19014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16b945f466c24e01bc710db136cde351. Latest consensus state: current_term: 1 leader_uuid: "16b945f466c24e01bc710db136cde351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16b945f466c24e01bc710db136cde351" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45963 } } }
I20260927 07:00:46.302917 19014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:46.304651 19019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:46.309835 19019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:46.319357 19019 catalog_manager.cc:1347] Generated new cluster ID: 345bb852a9374d3cb7a730f3f6735976
I20260927 07:00:46.319482 19019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:46.342989 19019 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:46.344342 19019 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:46.351572 19019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16b945f466c24e01bc710db136cde351: Generated new TSK 0
I20260927 07:00:46.352525 19019 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 07:00:46.389851 19037 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260927 07:00:46.390054 19037 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 07:00:46.391181 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:46.370746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48686 (local address 127.13.121.254:45963)
0927 07:00:46.370965 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:46.370972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:46.371350 (+ 378us) server_negotiation.cc:407] Connection header received
0927 07:00:46.371719 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:46.371728 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:46.371810 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:46.371942 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0927 07:00:46.372485 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:46.373224 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:46.374615 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:46.375074 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:46.375216 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:46.375764 (+ 548us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 07:00:46.390480 (+ 14716us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 07:00:46.390829 (+ 349us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48686: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":79}
W20260927 07:00:46.391314 19010 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48686: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260927 07:00:46.392642 19036 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:00:46.370563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:45963 (local address 127.0.0.1:48686)
0927 07:00:46.371248 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:46.371264 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:46.371424 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:46.371988 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:46.371995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:46.372008 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0927 07:00:46.372334 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:46.372343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:46.373362 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:46.373367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:46.374472 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:46.374483 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:46.375271 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:46.375279 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:46.375425 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:46.375491 (+ 66us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 07:00:46.390770 (+ 15279us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 07:00:46.391119 (+ 349us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.121.254:45963: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":326,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.121.254:45963: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260927 07:00:46.394486 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 18941
I20260927 07:00:46.396116 19002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:46.489871 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 18941
2026-09-27T07:00:46Z chronyd exiting
I20260927 07:00:46.537547 13799 test_util.cc:175] -----------------------------------------------
I20260927 07:00:46.537708 13799 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790492406663968-13799-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (742 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260927 07:00:46.541625 13799 mini_oidc.cc:152] Starting JWKS server
I20260927 07:00:46.543623 13799 webserver.cc:460] Webserver started at https://127.0.0.1:36823/ using document root <none> and password file <none>
I20260927 07:00:46.543855 13799 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 07:00:46.544652 13799 webserver.cc:460] Webserver started at http://127.0.0.1:42985/ using document root <none> and password file <none>
2026-09-27T07:00:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:46Z Disabled control of system clock
I20260927 07:00:46.571530 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:37707
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:37707
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36823/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/testchainca.cert with env {}
W20260927 07:00:46.892433 19044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:46.892778 19044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:46.892900 19044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:46.900528 19044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:46.900688 19044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:46.900753 19044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:46.900830 19044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:46.900890 19044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 07:00:46.900985 19044 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 07:00:46.901039 19044 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36823/jwks.json
I20260927 07:00:46.911767 19044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:37707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:37707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36823/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:46.913136 19044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:46.914453 19044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:46.926479 19050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:46.926415 19052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:46.926553 19049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:46.927418 19044 server_base.cc:1034] running on GCE node
I20260927 07:00:46.928306 19044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:46.929934 19044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:46.931207 19044 hybrid_clock.cc:648] HybridClock initialized: now 1790492446931117 us; error 43 us; skew 500 ppm
I20260927 07:00:46.931619 19044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:00:46.934496 19044 webserver.cc:460] Webserver started at http://127.13.121.254:37477/ using document root <none> and password file <none>
I20260927 07:00:46.935369 19044 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:46.935501 19044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:46.935833 19044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:46.938943 19044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "3320d746a4be4501ba6640f86afaa26d"
format_stamp: "Formatted at 2026-09-27 07:00:46 on dist-test-slave-p6hz"
server_key: "d1a6c12f891ef3d5079c1baa70530ac0"
server_key_iv: "12ef8226be56b7a03793f319c5af1cb8"
server_key_version: "encryptionkey@0"
I20260927 07:00:46.939890 19044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "3320d746a4be4501ba6640f86afaa26d"
format_stamp: "Formatted at 2026-09-27 07:00:46 on dist-test-slave-p6hz"
server_key: "d1a6c12f891ef3d5079c1baa70530ac0"
server_key_iv: "12ef8226be56b7a03793f319c5af1cb8"
server_key_version: "encryptionkey@0"
I20260927 07:00:46.945533 19044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260927 07:00:46.949285 19058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:46.950982 19044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:00:46.951242 19044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "3320d746a4be4501ba6640f86afaa26d"
format_stamp: "Formatted at 2026-09-27 07:00:46 on dist-test-slave-p6hz"
server_key: "d1a6c12f891ef3d5079c1baa70530ac0"
server_key_iv: "12ef8226be56b7a03793f319c5af1cb8"
server_key_version: "encryptionkey@0"
I20260927 07:00:46.951471 19044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:46.965801 19044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:46.966688 19044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:46.967001 19044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:46.984429 19044 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:37707
I20260927 07:00:46.984452 19109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:37707 every 8 connection(s)
I20260927 07:00:46.985798 19044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:46.989745 19110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:46.989931 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 19044
I20260927 07:00:46.990110 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:46.990890 13799 external_mini_cluster.cc:1428] Setting key fb8ceb05a334d9ff2db631805a7920ea
I20260927 07:00:47.003756 19110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d: Bootstrap starting.
I20260927 07:00:47.005698 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:46.992825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.13.121.254:37707)
0927 07:00:46.993887 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:46.993900 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:46.993956 (+ 56us) server_negotiation.cc:407] Connection header received
0927 07:00:46.995996 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:46.996031 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:46.996602 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:46.997006 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:46.997619 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:46.998926 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.000881 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.001679 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.001842 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:47.003223 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:47.003233 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:47.003250 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:47.003827 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:47.003937 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:47.003961 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:47.004052 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:47.004335 (+ 283us) server_negotiation.cc:299] Negotiation successful
0927 07:00:47.004673 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":364,"threads_started":1}
I20260927 07:00:47.007177 19110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:47.008257 19110 log.cc:824] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:47.009495 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:00:47.011129 19110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d: No bootstrap required, opened a new log
I20260927 07:00:47.018252 19110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320d746a4be4501ba6640f86afaa26d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37707 } }
I20260927 07:00:47.018622 19110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:47.018715 19110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3320d746a4be4501ba6640f86afaa26d, State: Initialized, Role: FOLLOWER
I20260927 07:00:47.019304 19110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320d746a4be4501ba6640f86afaa26d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37707 } }
I20260927 07:00:47.019526 19110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:47.019649 19110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:47.019840 19110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:47.021965 19110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320d746a4be4501ba6640f86afaa26d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37707 } }
I20260927 07:00:47.022519 19110 leader_election.cc:302] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3320d746a4be4501ba6640f86afaa26d; no voters:
I20260927 07:00:47.023191 19110 leader_election.cc:288] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:47.023334 19115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:47.023962 19115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [term 1 LEADER]: Becoming Leader. State: Replica: 3320d746a4be4501ba6640f86afaa26d, State: Running, Role: LEADER
I20260927 07:00:47.024467 19115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320d746a4be4501ba6640f86afaa26d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37707 } }
I20260927 07:00:47.025213 19110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:47.027398 19117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3320d746a4be4501ba6640f86afaa26d. Latest consensus state: current_term: 1 leader_uuid: "3320d746a4be4501ba6640f86afaa26d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320d746a4be4501ba6640f86afaa26d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37707 } } }
I20260927 07:00:47.027814 19117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:47.028929 19116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3320d746a4be4501ba6640f86afaa26d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320d746a4be4501ba6640f86afaa26d" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 37707 } } }
I20260927 07:00:47.029273 19116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:47.031531 19122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:47.036576 19122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:47.046111 19122 catalog_manager.cc:1347] Generated new cluster ID: d965fcd0577e447287f52430290a882e
I20260927 07:00:47.046233 19122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:47.066967 19122 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:47.068902 19122 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:47.077495 19122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3320d746a4be4501ba6640f86afaa26d: Generated new TSK 0
I20260927 07:00:47.078783 19122 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 07:00:47.121515 19140 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260927 07:00:47.121634 19140 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 07:00:47.122932 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:47.102698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48066 (local address 127.13.121.254:37707)
0927 07:00:47.102898 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:47.102905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:47.103391 (+ 486us) server_negotiation.cc:407] Connection header received
0927 07:00:47.103646 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:47.103650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:47.103735 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:47.103890 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0927 07:00:47.104404 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.105174 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.106644 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.107110 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.107273 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:47.107798 (+ 525us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 07:00:47.122177 (+ 14379us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 07:00:47.122605 (+ 428us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":66}
W20260927 07:00:47.123052 19139 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:00:47.102577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:37707 (local address 127.0.0.1:48066)
0927 07:00:47.103279 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:47.103297 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:47.103458 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:47.103924 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:47.103931 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:47.103944 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0927 07:00:47.104234 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:47.104245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.105328 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.105333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:47.106477 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:47.106489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.107319 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.107326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:47.107486 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:47.107525 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 07:00:47.122468 (+ 14943us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 07:00:47.122700 (+ 232us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.121.254:37707: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":391,"threads_started":1}
W20260927 07:00:47.123059 19113 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260927 07:00:47.124130 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 19044
I20260927 07:00:47.125427 19105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:47.219499 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 19044
2026-09-27T07:00:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (730 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260927 07:00:47.271646 13799 mini_oidc.cc:152] Starting JWKS server
I20260927 07:00:47.273500 13799 webserver.cc:460] Webserver started at https://127.0.0.1:44559/ using document root <none> and password file <none>
I20260927 07:00:47.273752 13799 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 07:00:47.274340 13799 webserver.cc:460] Webserver started at http://127.0.0.1:44879/ using document root <none> and password file <none>
2026-09-27T07:00:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:47Z Disabled control of system clock
I20260927 07:00:47.298480 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:40649
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:37833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:40649
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44559/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 07:00:47.621809 19147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:47.622155 19147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:47.622284 19147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:47.629735 19147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:47.629894 19147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:47.629962 19147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:47.630043 19147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:47.630105 19147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 07:00:47.630191 19147 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 07:00:47.630242 19147 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44559/jwks.json
I20260927 07:00:47.639819 19147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:37833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:40649
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:40649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44559/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:47.641148 19147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:47.642963 19147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:47.654765 19153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:47.654731 19155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:47.654793 19152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:47.655529 19147 server_base.cc:1034] running on GCE node
I20260927 07:00:47.656368 19147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:47.657905 19147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:47.659179 19147 hybrid_clock.cc:648] HybridClock initialized: now 1790492447659108 us; error 39 us; skew 500 ppm
I20260927 07:00:47.659574 19147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:00:47.662250 19147 webserver.cc:460] Webserver started at http://127.13.121.254:46591/ using document root <none> and password file <none>
I20260927 07:00:47.663002 19147 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:47.663108 19147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:47.663448 19147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:47.666355 19147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "336ef1cccdad4409b142294c2e717850"
format_stamp: "Formatted at 2026-09-27 07:00:47 on dist-test-slave-p6hz"
server_key: "a708bc4e31a9b52d19c565aad59c4a8f"
server_key_iv: "1e5d5068d4c4224807496e0d49ed14a1"
server_key_version: "encryptionkey@0"
I20260927 07:00:47.667129 19147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "336ef1cccdad4409b142294c2e717850"
format_stamp: "Formatted at 2026-09-27 07:00:47 on dist-test-slave-p6hz"
server_key: "a708bc4e31a9b52d19c565aad59c4a8f"
server_key_iv: "1e5d5068d4c4224807496e0d49ed14a1"
server_key_version: "encryptionkey@0"
I20260927 07:00:47.672304 19147 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260927 07:00:47.675935 19161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:47.677367 19147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:47.677542 19147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "336ef1cccdad4409b142294c2e717850"
format_stamp: "Formatted at 2026-09-27 07:00:47 on dist-test-slave-p6hz"
server_key: "a708bc4e31a9b52d19c565aad59c4a8f"
server_key_iv: "1e5d5068d4c4224807496e0d49ed14a1"
server_key_version: "encryptionkey@0"
I20260927 07:00:47.677747 19147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:47.697222 19147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:47.698087 19147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:47.698372 19147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:47.714524 19147 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:40649
I20260927 07:00:47.714530 19212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:40649 every 8 connection(s)
I20260927 07:00:47.715829 19147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:47.716679 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 19147
I20260927 07:00:47.716894 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:47.717446 13799 external_mini_cluster.cc:1428] Setting key 8d2296641b839f0733ef4f80ffb660a5
I20260927 07:00:47.720139 19213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:47.729979 19216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:47.719283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37238 (local address 127.13.121.254:40649)
0927 07:00:47.720215 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:47.720230 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:47.720279 (+ 49us) server_negotiation.cc:407] Connection header received
0927 07:00:47.721771 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:47.721791 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:47.722240 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:47.722690 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:47.723261 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.724547 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.725779 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.726422 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.726595 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:47.727703 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:47.727710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:47.727728 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:47.728310 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:47.728375 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:47.728398 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:47.728517 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:47.728730 (+ 213us) server_negotiation.cc:299] Negotiation successful
0927 07:00:47.729032 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":319,"threads_started":1}
I20260927 07:00:47.730762 19213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850: Bootstrap starting.
I20260927 07:00:47.734136 19213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:47.734181 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:00:47.735081 19213 log.cc:824] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:47.737872 19213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850: No bootstrap required, opened a new log
I20260927 07:00:47.744940 19213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "336ef1cccdad4409b142294c2e717850" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40649 } }
I20260927 07:00:47.745369 19213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:47.745484 19213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 336ef1cccdad4409b142294c2e717850, State: Initialized, Role: FOLLOWER
I20260927 07:00:47.746068 19213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "336ef1cccdad4409b142294c2e717850" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40649 } }
I20260927 07:00:47.746276 19213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:47.746395 19213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:47.746549 19213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:47.748777 19213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "336ef1cccdad4409b142294c2e717850" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40649 } }
I20260927 07:00:47.749291 19213 leader_election.cc:302] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 336ef1cccdad4409b142294c2e717850; no voters:
I20260927 07:00:47.749884 19213 leader_election.cc:288] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:47.750043 19218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:47.750710 19218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [term 1 LEADER]: Becoming Leader. State: Replica: 336ef1cccdad4409b142294c2e717850, State: Running, Role: LEADER
I20260927 07:00:47.751341 19218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "336ef1cccdad4409b142294c2e717850" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40649 } }
I20260927 07:00:47.752221 19213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:47.754092 19219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "336ef1cccdad4409b142294c2e717850" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "336ef1cccdad4409b142294c2e717850" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40649 } } }
I20260927 07:00:47.754560 19219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:47.755717 19220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 336ef1cccdad4409b142294c2e717850. Latest consensus state: current_term: 1 leader_uuid: "336ef1cccdad4409b142294c2e717850" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "336ef1cccdad4409b142294c2e717850" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 40649 } } }
I20260927 07:00:47.756127 19220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:47.757692 19227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:47.763724 19227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:47.772365 19227 catalog_manager.cc:1347] Generated new cluster ID: 39cba764238d4b70a79b34b617edb846
I20260927 07:00:47.772496 19227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:47.788579 19227 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:47.789884 19227 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:47.795650 19227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 336ef1cccdad4409b142294c2e717850: Generated new TSK 0
I20260927 07:00:47.796638 19227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:00:47.827736 19216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:47.819419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37252 (local address 127.13.121.254:40649)
0927 07:00:47.819615 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:47.819621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:47.819924 (+ 303us) server_negotiation.cc:407] Connection header received
0927 07:00:47.820123 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:47.820129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:47.820243 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:47.820405 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0927 07:00:47.820845 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.821942 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.823388 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.824090 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.824220 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:47.824723 (+ 503us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 07:00:47.827081 (+ 2358us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 07:00:47.827435 (+ 354us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37252: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":74}
W20260927 07:00:47.827869 19216 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37252: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260927 07:00:47.829473 19242 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:00:47.819250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:40649 (local address 127.0.0.1:37252)
0927 07:00:47.819819 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:47.819835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:47.819958 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:47.820450 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:47.820454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:47.820465 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0927 07:00:47.820713 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:47.820720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.822101 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.822105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:47.823248 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:47.823262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:47.824265 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:47.824270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:47.824408 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:47.824443 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 07:00:47.827308 (+ 2865us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 07:00:47.829038 (+ 1730us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.121.254:40649: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":1400,"thread_start_us":303,"threads_started":1}
I20260927 07:00:47.830863 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 19147
I20260927 07:00:47.832278 19208 generic_service.cc:196] Checking for leaks (request via RPC)
E20260927 07:00:47.839118 19243 webserver.cc:533] Webserver: sslize failed:
E20260927 07:00:47.839268 19243 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 07:00:47.922797 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 19147
2026-09-27T07:00:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (702 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-27T07:00:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:47Z Disabled control of system clock
I20260927 07:00:47.999501 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:44263
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:44263
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/testchainca.cert with env {}
W20260927 07:00:48.328377 19248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:48.328682 19248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:48.328790 19248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:48.336010 19248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:48.336133 19248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:48.336170 19248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:48.336247 19248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:48.336303 19248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 07:00:48.336386 19248 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 07:00:48.336443 19248 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260927 07:00:48.346505 19248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:44263
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:44263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:48.347883 19248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:48.349176 19248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:48.360971 19253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:48.361171 19256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:48.361014 19254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:48.361878 19248 server_base.cc:1034] running on GCE node
I20260927 07:00:48.362820 19248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:48.364488 19248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:48.365738 19248 hybrid_clock.cc:648] HybridClock initialized: now 1790492448365685 us; error 51 us; skew 500 ppm
I20260927 07:00:48.366137 19248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:00:48.368692 19248 webserver.cc:460] Webserver started at http://127.13.121.254:33345/ using document root <none> and password file <none>
I20260927 07:00:48.369414 19248 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:48.369520 19248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:48.369809 19248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:48.372754 19248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "d06cf74bcc1d4c49a483551a02a11b92"
format_stamp: "Formatted at 2026-09-27 07:00:48 on dist-test-slave-p6hz"
server_key: "acd092929919c91a893c8fa5a9520399"
server_key_iv: "64baafe89547a3c50d5abe241bf47286"
server_key_version: "encryptionkey@0"
I20260927 07:00:48.373534 19248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "d06cf74bcc1d4c49a483551a02a11b92"
format_stamp: "Formatted at 2026-09-27 07:00:48 on dist-test-slave-p6hz"
server_key: "acd092929919c91a893c8fa5a9520399"
server_key_iv: "64baafe89547a3c50d5abe241bf47286"
server_key_version: "encryptionkey@0"
I20260927 07:00:48.379035 19248 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260927 07:00:48.382710 19262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:48.384298 19248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 07:00:48.384479 19248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "d06cf74bcc1d4c49a483551a02a11b92"
format_stamp: "Formatted at 2026-09-27 07:00:48 on dist-test-slave-p6hz"
server_key: "acd092929919c91a893c8fa5a9520399"
server_key_iv: "64baafe89547a3c50d5abe241bf47286"
server_key_version: "encryptionkey@0"
I20260927 07:00:48.384687 19248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:48.409569 19248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:48.410506 19248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:48.410776 19248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:48.427922 19248 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:44263
I20260927 07:00:48.427939 19313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:44263 every 8 connection(s)
I20260927 07:00:48.429319 19248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:48.433246 19314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:48.439240 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 19248
I20260927 07:00:48.439405 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:48.440026 13799 external_mini_cluster.cc:1428] Setting key 86fab8b8b333e330a316a58f837829b3
I20260927 07:00:48.444417 19314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92: Bootstrap starting.
I20260927 07:00:48.447726 19314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:48.449090 19314 log.cc:824] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:48.452903 19314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92: No bootstrap required, opened a new log
I20260927 07:00:48.455056 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:48.441998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.13.121.254:44263)
0927 07:00:48.443036 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:48.443049 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:48.443095 (+ 46us) server_negotiation.cc:407] Connection header received
0927 07:00:48.444846 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:48.444874 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:48.445403 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:48.445824 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:48.446427 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:48.447780 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:48.449629 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:48.450804 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:48.450969 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:48.452280 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:48.452291 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:48.452308 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:00:48.452858 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:48.452943 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:00:48.452973 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:48.453096 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:48.453371 (+ 275us) server_negotiation.cc:299] Negotiation successful
0927 07:00:48.453742 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":339,"threads_started":1}
I20260927 07:00:48.459357 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:00:48.462015 19314 raft_consensus.cc:348] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06cf74bcc1d4c49a483551a02a11b92" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44263 } }
I20260927 07:00:48.462530 19314 raft_consensus.cc:374] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:48.462641 19314 raft_consensus.cc:729] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d06cf74bcc1d4c49a483551a02a11b92, State: Initialized, Role: FOLLOWER
I20260927 07:00:48.463327 19314 consensus_queue.cc:260] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06cf74bcc1d4c49a483551a02a11b92" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44263 } }
I20260927 07:00:48.463563 19314 raft_consensus.cc:388] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:48.463701 19314 raft_consensus.cc:482] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:48.463863 19314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:48.466009 19314 raft_consensus.cc:504] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06cf74bcc1d4c49a483551a02a11b92" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44263 } }
I20260927 07:00:48.466601 19314 leader_election.cc:302] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d06cf74bcc1d4c49a483551a02a11b92; no voters:
I20260927 07:00:48.467291 19314 leader_election.cc:288] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:48.467419 19319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:48.468039 19319 raft_consensus.cc:686] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [term 1 LEADER]: Becoming Leader. State: Replica: d06cf74bcc1d4c49a483551a02a11b92, State: Running, Role: LEADER
I20260927 07:00:48.468622 19319 consensus_queue.cc:237] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06cf74bcc1d4c49a483551a02a11b92" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44263 } }
I20260927 07:00:48.469648 19314 sys_catalog.cc:564] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:48.472575 19321 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d06cf74bcc1d4c49a483551a02a11b92. Latest consensus state: current_term: 1 leader_uuid: "d06cf74bcc1d4c49a483551a02a11b92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06cf74bcc1d4c49a483551a02a11b92" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44263 } } }
I20260927 07:00:48.472623 19320 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d06cf74bcc1d4c49a483551a02a11b92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06cf74bcc1d4c49a483551a02a11b92" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44263 } } }
I20260927 07:00:48.472962 19321 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:48.472998 19320 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:48.474821 19328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:48.480101 19328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:48.489315 19328 catalog_manager.cc:1347] Generated new cluster ID: 5c6660e403614105af4a6d6c2530ab0a
I20260927 07:00:48.489452 19328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:48.520500 19328 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:48.521829 19328 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:48.528297 19328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d06cf74bcc1d4c49a483551a02a11b92: Generated new TSK 0
I20260927 07:00:48.529284 19328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:00:48.609460 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:48.553039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55924 (local address 127.13.121.254:44263)
0927 07:00:48.553244 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:48.553250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:48.553582 (+ 332us) server_negotiation.cc:407] Connection header received
0927 07:00:48.553822 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:48.553826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:48.553900 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:48.554026 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0927 07:00:48.554578 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:48.555508 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:48.556959 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:48.557429 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:48.557548 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:48.558053 (+ 505us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 07:00:48.608718 (+ 50665us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 07:00:48.609116 (+ 398us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55924: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":81}
W20260927 07:00:48.609601 19317 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55924: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260927 07:00:48.609706 19343 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:00:48.552789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44263 (local address 127.0.0.1:55924)
0927 07:00:48.553469 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:48.553490 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:48.553651 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:48.554064 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:48.554070 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:48.554084 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0927 07:00:48.554384 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:48.554396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:48.555664 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:48.555671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:48.556797 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:48.556811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:48.557605 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:48.557613 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:48.557747 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:48.557786 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 07:00:48.609025 (+ 51239us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 07:00:48.609308 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.121.254:44263: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":332,"threads_started":1}
I20260927 07:00:48.611318 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 19248
I20260927 07:00:48.612515 19309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:48.710702 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 19248
2026-09-27T07:00:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (767 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (400 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (392 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:49 dist-test-slave-p6hz krb5kdc[19358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:49 dist-test-slave-p6hz krb5kdc[19358](info): set up 2 sockets
Sep 27 07:00:49 dist-test-slave-p6hz krb5kdc[19358](info): commencing operation
krb5kdc: starting...
W20260927 07:00:50.519654 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:50 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:00:50.719574 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:35641
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:35641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:51.040824 19371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:51.041167 19371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:51.041293 19371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:51.049289 19371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:51.049424 19371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:51.049481 19371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:51.049562 19371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:51.049623 19371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:51.061525 19371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:35641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:35641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:51.063005 19371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:51.064616 19371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:51.077310 19379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:51.077355 19376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:51.077330 19377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:51.078059 19371 server_base.cc:1034] running on GCE node
I20260927 07:00:51.079505 19371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:51.081166 19371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:51.082442 19371 hybrid_clock.cc:648] HybridClock initialized: now 1790492451082348 us; error 69 us; skew 500 ppm
I20260927 07:00:51.082845 19371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:51 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:51.086246 19371 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:00:51.088035 19371 webserver.cc:460] Webserver started at http://127.13.121.254:34255/ using document root <none> and password file <none>
I20260927 07:00:51.088758 19371 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:51.088871 19371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:51.089154 19371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:51.092383 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "c3b8e72e360e419eacbdb0bbb5b972d0"
format_stamp: "Formatted at 2026-09-27 07:00:51 on dist-test-slave-p6hz"
server_key: "744a94709ecec1f995249fad619a2d54"
server_key_iv: "7970989722f86e453736e8d5684845e7"
server_key_version: "encryptionkey@0"
I20260927 07:00:51.093238 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "c3b8e72e360e419eacbdb0bbb5b972d0"
format_stamp: "Formatted at 2026-09-27 07:00:51 on dist-test-slave-p6hz"
server_key: "744a94709ecec1f995249fad619a2d54"
server_key_iv: "7970989722f86e453736e8d5684845e7"
server_key_version: "encryptionkey@0"
I20260927 07:00:51.100528 19371 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260927 07:00:51.105474 19386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:51.107244 19371 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 07:00:51.107445 19371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "c3b8e72e360e419eacbdb0bbb5b972d0"
format_stamp: "Formatted at 2026-09-27 07:00:51 on dist-test-slave-p6hz"
server_key: "744a94709ecec1f995249fad619a2d54"
server_key_iv: "7970989722f86e453736e8d5684845e7"
server_key_version: "encryptionkey@0"
I20260927 07:00:51.107658 19371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:51.139911 19371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:51.144806 19371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:51.145251 19371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:51.165766 19371 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:35641
I20260927 07:00:51.165786 19437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:35641 every 8 connection(s)
I20260927 07:00:51.167272 19371 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:51.168255 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 19371
I20260927 07:00:51.168483 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:51.169067 13799 external_mini_cluster.cc:1428] Setting key 5e60be5ab4e4ebd3bf0eb5874bb0077e
I20260927 07:00:51.171855 19438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 07:00:51 dist-test-slave-p6hz krb5kdc[19358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:51.186050 19438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0: Bootstrap starting.
I20260927 07:00:51.189656 19438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:51.190831 19438 log.cc:824] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:51.194671 19438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0: No bootstrap required, opened a new log
I20260927 07:00:51.196525 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:51.172039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60818 (local address 127.13.121.254:35641)
0927 07:00:51.172936 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:51.172949 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:51.172994 (+ 45us) server_negotiation.cc:407] Connection header received
0927 07:00:51.174571 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:51.174595 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:51.175060 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:51.175520 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:51.177432 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:51.178825 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:51.180665 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:51.181592 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:51.181906 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:51.187205 (+ 5299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:51.187219 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:51.187239 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:51.187288 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:51.190437 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:51.191451 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:51.191462 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:51.191472 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:51.191758 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:51.192166 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:51.192171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:51.192174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:51.193362 (+ 1188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:51.193643 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:51.194255 (+ 612us) server_negotiation.cc:299] Negotiation successful
0927 07:00:51.194850 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":340,"threads_started":1}
I20260927 07:00:51.203783 19438 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8e72e360e419eacbdb0bbb5b972d0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35641 } }
I20260927 07:00:51.204365 19438 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:51.204491 19438 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b8e72e360e419eacbdb0bbb5b972d0, State: Initialized, Role: FOLLOWER
I20260927 07:00:51.205168 19438 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8e72e360e419eacbdb0bbb5b972d0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35641 } }
I20260927 07:00:51.205402 19438 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:51.205533 19438 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:51.205685 19438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:51.207890 19438 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8e72e360e419eacbdb0bbb5b972d0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35641 } }
I20260927 07:00:51.208410 19438 leader_election.cc:302] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b8e72e360e419eacbdb0bbb5b972d0; no voters:
I20260927 07:00:51.208978 19438 leader_election.cc:288] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:51.209086 19443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:51.210079 19443 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [term 1 LEADER]: Becoming Leader. State: Replica: c3b8e72e360e419eacbdb0bbb5b972d0, State: Running, Role: LEADER
I20260927 07:00:51.210765 19443 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8e72e360e419eacbdb0bbb5b972d0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35641 } }
I20260927 07:00:51.211210 19438 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:51.213560 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b8e72e360e419eacbdb0bbb5b972d0. Latest consensus state: current_term: 1 leader_uuid: "c3b8e72e360e419eacbdb0bbb5b972d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8e72e360e419eacbdb0bbb5b972d0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35641 } } }
I20260927 07:00:51.213649 19444 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b8e72e360e419eacbdb0bbb5b972d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b8e72e360e419eacbdb0bbb5b972d0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35641 } } }
I20260927 07:00:51.214026 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:51.214063 19444 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0 [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:51.216342 19448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:51.222141 19448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:51.231273 19448 catalog_manager.cc:1347] Generated new cluster ID: 0d56764214e44b6eb5fb45d3936660e7
I20260927 07:00:51.231406 19448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:51.256745 19448 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:51.258898 19448 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:51.267908 19448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b8e72e360e419eacbdb0bbb5b972d0: Generated new TSK 0
I20260927 07:00:51.269057 19448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:00:51.323791 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35641
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:51.680641 19464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:51.680958 19464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:51.681061 19464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:51.688676 19464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:51.688814 19464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:51.688956 19464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:51.700673 19464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:51.702553 19464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:51.704447 19464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:51.722117 19469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:51.722174 19472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:51.722200 19470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:51.723026 19464 server_base.cc:1034] running on GCE node
I20260927 07:00:51.723798 19464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:51.724941 19464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:51.726230 19464 hybrid_clock.cc:648] HybridClock initialized: now 1790492451726158 us; error 62 us; skew 500 ppm
I20260927 07:00:51.726648 19464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:51 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:51.730465 19464 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:00:51.732231 19464 webserver.cc:460] Webserver started at http://127.13.121.193:42363/ using document root <none> and password file <none>
I20260927 07:00:51.733000 19464 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:51.733130 19464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:51.734047 19464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:51.737448 19464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "229ec5de89034663b9d1af10f31a43c1"
format_stamp: "Formatted at 2026-09-27 07:00:51 on dist-test-slave-p6hz"
server_key: "3431772cc6aa69273cb9a5a9952f9361"
server_key_iv: "6515ff2492b7ae125ce078baf2438143"
server_key_version: "encryptionkey@0"
I20260927 07:00:51.738309 19464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "229ec5de89034663b9d1af10f31a43c1"
format_stamp: "Formatted at 2026-09-27 07:00:51 on dist-test-slave-p6hz"
server_key: "3431772cc6aa69273cb9a5a9952f9361"
server_key_iv: "6515ff2492b7ae125ce078baf2438143"
server_key_version: "encryptionkey@0"
I20260927 07:00:51.744547 19464 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260927 07:00:51.749128 19479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:51.751255 19464 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260927 07:00:51.751581 19464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "229ec5de89034663b9d1af10f31a43c1"
format_stamp: "Formatted at 2026-09-27 07:00:51 on dist-test-slave-p6hz"
server_key: "3431772cc6aa69273cb9a5a9952f9361"
server_key_iv: "6515ff2492b7ae125ce078baf2438143"
server_key_version: "encryptionkey@0"
I20260927 07:00:51.751828 19464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:51.765889 19464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:51.772284 19464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:51.772595 19464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:51.773612 19464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:51.775046 19464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:51.775128 19464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:51.775277 19464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:51.775331 19464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:51.807458 19464 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:37921
I20260927 07:00:51.807512 19591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:37921 every 8 connection(s)
I20260927 07:00:51.808703 19464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:51.817329 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 19464
I20260927 07:00:51.817520 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:51.818202 13799 external_mini_cluster.cc:1428] Setting key 1e1b5d06ec80430d16938f83bf05b94b
Sep 27 07:00:51 dist-test-slave-p6hz krb5kdc[19358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:51.829658 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:51.811299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:48325 (local address 127.13.121.254:35641)
0927 07:00:51.811774 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:51.811778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:51.812885 (+ 1107us) server_negotiation.cc:407] Connection header received
0927 07:00:51.814687 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:51.814692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:51.814775 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:51.814871 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:51.816504 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:51.817537 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:51.819362 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:51.819801 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:51.819917 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:51.824011 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:51.824017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:51.824020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:51.824056 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:51.826382 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:51.827379 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:51.827383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:51.827386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:51.827511 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:51.827946 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:51.827949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:51.827951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:51.828211 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:51.828326 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:51.829219 (+ 893us) server_negotiation.cc:299] Negotiation successful
0927 07:00:51.829407 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":255,"threads_started":1}
I20260927 07:00:51.831010 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:51.811914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:35641 (local address 127.13.121.193:48325)
0927 07:00:51.812715 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:51.812765 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:51.814420 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:51.815120 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:51.815129 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:51.815707 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:51.816324 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:51.816341 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:51.817688 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:51.817694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:51.819221 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:51.819246 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:51.820037 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:51.820046 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:51.820286 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:51.820981 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:51.821005 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:51.823677 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:51.826550 (+ 2873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:51.826560 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:51.826579 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:51.827203 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:51.827611 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:51.827615 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:51.827618 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:51.827841 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:51.828325 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:51.828332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:51.828802 (+ 470us) client_negotiation.cc:769] Sending connection context
0927 07:00:51.829099 (+ 297us) client_negotiation.cc:240] Negotiation successful
0927 07:00:51.829625 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":324,"threads_started":1}
I20260927 07:00:51.832643 19592 heartbeater.cc:347] Connected to a master server at 127.13.121.254:35641
I20260927 07:00:51.832998 19592 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:51.833829 19592 heartbeater.cc:511] Master 127.13.121.254:35641 requested a full tablet report, sending...
I20260927 07:00:51.836509 19403 ts_manager.cc:194] Registered new tserver with Master: 229ec5de89034663b9d1af10f31a43c1 (127.13.121.193:37921)
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:00:51.838577 19403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:48325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:00:51.866799 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35641
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:52.205271 19598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:52.205566 19598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:52.205659 19598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:52.213673 19598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:52.213809 19598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:52.213930 19598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:52.224120 19598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:52.225461 19598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:52.226719 19598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:52.239647 19604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:52.239606 19606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:52.239643 19603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:52.240206 19598 server_base.cc:1034] running on GCE node
I20260927 07:00:52.240970 19598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:52.241935 19598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:52.243184 19598 hybrid_clock.cc:648] HybridClock initialized: now 1790492452243103 us; error 45 us; skew 500 ppm
I20260927 07:00:52.243516 19598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:52.246603 19598 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:00:52.248111 19598 webserver.cc:460] Webserver started at http://127.13.121.194:34787/ using document root <none> and password file <none>
I20260927 07:00:52.248801 19598 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:52.248921 19598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:52.249212 19598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:52.252239 19598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "37ec42cc075e4e18b151b2192a7af28d"
format_stamp: "Formatted at 2026-09-27 07:00:52 on dist-test-slave-p6hz"
server_key: "0b2777c152b301300e4c1d7949983cd6"
server_key_iv: "d051ca61d41e2b0642f35a03448680aa"
server_key_version: "encryptionkey@0"
I20260927 07:00:52.253080 19598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "37ec42cc075e4e18b151b2192a7af28d"
format_stamp: "Formatted at 2026-09-27 07:00:52 on dist-test-slave-p6hz"
server_key: "0b2777c152b301300e4c1d7949983cd6"
server_key_iv: "d051ca61d41e2b0642f35a03448680aa"
server_key_version: "encryptionkey@0"
I20260927 07:00:52.259022 19598 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 07:00:52.262873 19613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:52.264359 19598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:00:52.264528 19598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "37ec42cc075e4e18b151b2192a7af28d"
format_stamp: "Formatted at 2026-09-27 07:00:52 on dist-test-slave-p6hz"
server_key: "0b2777c152b301300e4c1d7949983cd6"
server_key_iv: "d051ca61d41e2b0642f35a03448680aa"
server_key_version: "encryptionkey@0"
I20260927 07:00:52.264734 19598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:52.282312 19598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:52.286775 19598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:52.287118 19598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:52.288172 19598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:52.289604 19598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:52.289674 19598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:52.289789 19598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:52.289845 19598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:52.318678 19598 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:37831
I20260927 07:00:52.318724 19725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:37831 every 8 connection(s)
I20260927 07:00:52.319937 19598 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:52.326547 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 19598
I20260927 07:00:52.326686 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:52.327103 13799 external_mini_cluster.cc:1428] Setting key 210d5deb78992b1a2466375363b216fc
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:52.341178 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:52.322378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:58899 (local address 127.13.121.254:35641)
0927 07:00:52.322567 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:52.322571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:52.323797 (+ 1226us) server_negotiation.cc:407] Connection header received
0927 07:00:52.325579 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:52.325584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:52.325663 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:52.325745 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:52.327302 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.328127 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.330174 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.330615 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.330711 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:52.335023 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:52.335029 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:52.335033 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:52.335069 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:52.338086 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.338872 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.338878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.338882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.339039 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.339470 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.339474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.339477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.339805 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:52.339936 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:52.340613 (+ 677us) server_negotiation.cc:299] Negotiation successful
0927 07:00:52.340833 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 07:00:52.342031 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:52.322842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:35641 (local address 127.13.121.194:58899)
0927 07:00:52.323633 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:52.323671 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:52.325324 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:52.325979 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:52.325988 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:52.326500 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:52.327102 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:52.327120 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.328274 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.328279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:52.330027 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:52.330044 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.330841 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.330847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:52.331176 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:52.331947 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:52.331990 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:52.334678 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:52.338268 (+ 3590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:52.338277 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:52.338293 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:52.338743 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:52.339161 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:52.339166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:52.339169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:52.339366 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:52.339921 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:52.339927 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:52.340267 (+ 340us) client_negotiation.cc:769] Sending connection context
0927 07:00:52.340496 (+ 229us) client_negotiation.cc:240] Negotiation successful
0927 07:00:52.340885 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":355,"threads_started":1}
I20260927 07:00:52.343433 19726 heartbeater.cc:347] Connected to a master server at 127.13.121.254:35641
I20260927 07:00:52.343770 19726 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:52.344705 19726 heartbeater.cc:511] Master 127.13.121.254:35641 requested a full tablet report, sending...
I20260927 07:00:52.346350 19403 ts_manager.cc:194] Registered new tserver with Master: 37ec42cc075e4e18b151b2192a7af28d (127.13.121.194:37831)
I20260927 07:00:52.347196 19403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:58899
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:00:52.379602 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35641
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:52.704672 19731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:52.704982 19731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:52.705116 19731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:52.712457 19731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:52.712598 19731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:52.712754 19731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:52.722679 19731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:52.724021 19731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:52.725299 19731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:52.737872 19736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:52.737797 19739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:52.737828 19737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:52.738665 19731 server_base.cc:1034] running on GCE node
I20260927 07:00:52.739303 19731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:52.740275 19731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:52.741492 19731 hybrid_clock.cc:648] HybridClock initialized: now 1790492452741406 us; error 71 us; skew 500 ppm
I20260927 07:00:52.741809 19731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:52.744860 19731 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:00:52.746256 19731 webserver.cc:460] Webserver started at http://127.13.121.195:41841/ using document root <none> and password file <none>
I20260927 07:00:52.746912 19731 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:52.747035 19731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:52.747360 19731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:52.750228 19731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "0c7a857acb65468bb0c4ff8dd4feb172"
format_stamp: "Formatted at 2026-09-27 07:00:52 on dist-test-slave-p6hz"
server_key: "873dbe9a24eaffe11192935f7a1def46"
server_key_iv: "1c2ed14d4d9abc02ca1faf1bb353a016"
server_key_version: "encryptionkey@0"
I20260927 07:00:52.751009 19731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "0c7a857acb65468bb0c4ff8dd4feb172"
format_stamp: "Formatted at 2026-09-27 07:00:52 on dist-test-slave-p6hz"
server_key: "873dbe9a24eaffe11192935f7a1def46"
server_key_iv: "1c2ed14d4d9abc02ca1faf1bb353a016"
server_key_version: "encryptionkey@0"
I20260927 07:00:52.756399 19731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:00:52.760092 19746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:52.761535 19731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:52.761700 19731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "0c7a857acb65468bb0c4ff8dd4feb172"
format_stamp: "Formatted at 2026-09-27 07:00:52 on dist-test-slave-p6hz"
server_key: "873dbe9a24eaffe11192935f7a1def46"
server_key_iv: "1c2ed14d4d9abc02ca1faf1bb353a016"
server_key_version: "encryptionkey@0"
I20260927 07:00:52.761933 19731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:52.787804 19731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:52.792086 19731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:52.792359 19731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:52.793329 19731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:52.794719 19731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:52.794797 19731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:52.794917 19731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:52.794967 19731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:52.825497 19731 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:46191
I20260927 07:00:52.825541 19858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:46191 every 8 connection(s)
I20260927 07:00:52.826725 19731 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:52.828665 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 19731
I20260927 07:00:52.828809 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:52.829291 13799 external_mini_cluster.cc:1428] Setting key ad1794b00ec0d5cb3bb8b9755037c56c
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:00:52.841748 19592 heartbeater.cc:503] Master 127.13.121.254:35641 was elected leader, sending a full tablet report...
I20260927 07:00:52.846541 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:52.829152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:49797 (local address 127.13.121.254:35641)
0927 07:00:52.829347 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:52.829353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:52.830656 (+ 1303us) server_negotiation.cc:407] Connection header received
0927 07:00:52.832886 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:52.832890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:52.832986 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:52.833069 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:52.834650 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.835450 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.837024 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.837413 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.837514 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:52.840794 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:52.840798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:52.840802 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:52.840832 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:52.843677 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.844502 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.844508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.844511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.844661 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.845070 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.845073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.845075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.845351 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:52.845462 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:52.846137 (+ 675us) server_negotiation.cc:299] Negotiation successful
0927 07:00:52.846296 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 07:00:52.847604 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:52.829675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:35641 (local address 127.13.121.195:49797)
0927 07:00:52.830496 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:52.830553 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:52.832634 (+ 2081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:52.833344 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:52.833354 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:52.833890 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:52.834499 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:52.834519 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.835584 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.835588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:52.836894 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:52.836920 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.837583 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.837588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:52.837790 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:52.838407 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:52.838429 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:52.840588 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:52.843809 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:52.843818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:52.843836 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:52.844381 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:52.844761 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:52.844764 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:52.844766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:52.844987 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:52.845470 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:52.845477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:52.845803 (+ 326us) client_negotiation.cc:769] Sending connection context
0927 07:00:52.846065 (+ 262us) client_negotiation.cc:240] Negotiation successful
0927 07:00:52.846480 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":332,"threads_started":1}
I20260927 07:00:52.848868 19859 heartbeater.cc:347] Connected to a master server at 127.13.121.254:35641
I20260927 07:00:52.849160 19859 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:52.849828 19859 heartbeater.cc:511] Master 127.13.121.254:35641 requested a full tablet report, sending...
I20260927 07:00:52.851305 19402 ts_manager.cc:194] Registered new tserver with Master: 0c7a857acb65468bb0c4ff8dd4feb172 (127.13.121.195:46191)
I20260927 07:00:52.852113 19402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:49797
I20260927 07:00:52.857656 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/test-admin.keytab.
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:52.885610 13799 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260927 07:00:52.890398 19866 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:00:52.888052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.13.121.193:37921)
0927 07:00:52.888740 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:52.888747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:52.888773 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:00:52.888838 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:52.888844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:52.889055 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:52.889181 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:52.889997 (+ 816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46412: BlockingRecv error: recv got EOF from 127.0.0.1:46412 (error 108)
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":401,"threads_started":1}
W20260927 07:00:52.890422 13799 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5cc' of type FILE
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:52.892077 13799 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:52.893427 13799 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:52.908525 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:52.895034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46422 (local address 127.13.121.193:37921)
0927 07:00:52.895256 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:52.895262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:52.895393 (+ 131us) server_negotiation.cc:407] Connection header received
0927 07:00:52.895545 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:52.895549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:52.895608 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:52.895687 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:52.896625 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.897603 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.898862 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.899532 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.899639 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:52.903039 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:52.903054 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:52.903071 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:52.903107 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:52.905852 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.906603 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.906612 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.906614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.906744 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.907054 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.907056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.907058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.907464 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:52.907680 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:52.907989 (+ 309us) server_negotiation.cc:299] Negotiation successful
0927 07:00:52.908214 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260927 07:00:52.914160 19866 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:00:52.912551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.13.121.193:37921)
0927 07:00:52.912740 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:52.912745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:52.912948 (+ 203us) server_negotiation.cc:407] Connection header received
0927 07:00:52.913106 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:52.913110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:52.913177 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:52.913252 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:52.913934 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46428: BlockingRecv error: recv got EOF from 127.0.0.1:46428 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260927 07:00:52.914407 13799 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5cc' of type FILE
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:52.915879 13799 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:52.917155 13799 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:52.931066 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:52.918680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46432 (local address 127.13.121.193:37921)
0927 07:00:52.918886 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:52.918892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:52.919096 (+ 204us) server_negotiation.cc:407] Connection header received
0927 07:00:52.919236 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:52.919240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:52.919300 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:52.919380 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:52.920317 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.921040 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.922304 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.922767 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.922918 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:52.926306 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:52.926312 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:52.926315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:52.926360 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:52.928715 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.929377 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.929381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.929384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.929516 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.929820 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.929823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.929825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.930090 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:52.930308 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:52.930601 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 07:00:52.930772 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260927 07:00:52.935570 19866 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:00:52.933873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.13.121.193:37921)
0927 07:00:52.934060 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:52.934066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:52.934311 (+ 245us) server_negotiation.cc:407] Connection header received
0927 07:00:52.934473 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:52.934478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:52.934549 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:52.934633 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:52.935366 (+ 733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46436: BlockingRecv error: recv got EOF from 127.0.0.1:46436 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260927 07:00:52.935858 13799 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790492406663968-13799-0/krb5kdc/krb5cc' of type FILE
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:52.937542 13799 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:52.938917 13799 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 07:00:52 dist-test-slave-p6hz krb5kdc[19358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:00:52.953256 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:52.940659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46438 (local address 127.13.121.193:37921)
0927 07:00:52.940902 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:52.940909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:52.941159 (+ 250us) server_negotiation.cc:407] Connection header received
0927 07:00:52.941372 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:52.941376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:52.941441 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:52.941530 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:52.942531 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.943405 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.944671 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.945087 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.945184 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:52.948518 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:52.948523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:52.948526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:52.948564 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:52.950923 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.951628 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.951632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.951634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.951759 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.952081 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.952084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.952086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.952336 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:52.952541 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:52.952758 (+ 217us) server_negotiation.cc:299] Negotiation successful
0927 07:00:52.952920 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260927 07:00:52.954931 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 19464
I20260927 07:00:52.968267 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:52.955721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46454 (local address 127.13.121.193:37921)
0927 07:00:52.955898 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:52.955902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:52.956148 (+ 246us) server_negotiation.cc:407] Connection header received
0927 07:00:52.956461 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:52.956466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:52.956539 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:52.956628 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:52.957642 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.958506 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.959792 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:52.960225 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:52.960332 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:52.962741 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:52.962746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:52.962750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:52.962796 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:52.965383 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.966091 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.966095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.966098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.966237 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:52.966971 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:52.966975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:52.966977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:52.967304 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:52.967522 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:52.967850 (+ 328us) server_negotiation.cc:299] Negotiation successful
0927 07:00:52.968019 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 07:00:52.968801 19587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:53.051934 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 19464
I20260927 07:00:53.066344 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 19598
Sep 27 07:00:53 dist-test-slave-p6hz krb5kdc[19358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:00:53.082056 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:53.067178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.13.121.194:37831)
0927 07:00:53.067698 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:53.067705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:53.067731 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:00:53.067796 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:53.067803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:53.068056 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:53.068189 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:53.069137 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:53.070230 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:53.071571 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:53.072137 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:53.072259 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:53.076314 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:53.076329 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:53.076346 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:53.076386 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:53.079231 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:53.079974 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:53.079982 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:53.079985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:53.080119 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:53.080471 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:53.080475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:53.080478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:53.080843 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:53.081137 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:53.081481 (+ 344us) server_negotiation.cc:299] Negotiation successful
0927 07:00:53.081756 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":294,"threads_started":1}
I20260927 07:00:53.083575 19721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:53.166762 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 19598
I20260927 07:00:53.182971 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 19731
Sep 27 07:00:53 dist-test-slave-p6hz krb5kdc[19358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:00:53.198589 19881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:53.183935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52244 (local address 127.13.121.195:46191)
0927 07:00:53.184560 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:53.184567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:53.184595 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:00:53.184652 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:53.184659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:53.184859 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:53.184981 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:53.185934 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:53.186941 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:53.188296 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:53.188873 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:53.189028 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:53.192961 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:53.192976 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:53.192993 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:53.193032 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:53.196030 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:53.196786 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:53.196794 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:53.196796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:53.196909 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:53.197219 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:53.197222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:53.197224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:53.197557 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:53.197787 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:53.198092 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 07:00:53.198298 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":328,"threads_started":1}
I20260927 07:00:53.199997 19854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:53.281188 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 19731
I20260927 07:00:53.293507 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 19371
I20260927 07:00:53.294663 19433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:53.395574 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 19371
2026-09-27T07:00:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3897 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:00:53 dist-test-slave-p6hz krb5kdc[19886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:00:53 dist-test-slave-p6hz krb5kdc[19886](info): set up 2 sockets
Sep 27 07:00:53 dist-test-slave-p6hz krb5kdc[19886](info): commencing operation
krb5kdc: starting...
W20260927 07:00:54.411859 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.003s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:00:54 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:00:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:00:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.254.keytab.
Entry for principal oryx/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.254.keytab.
I20260927 07:00:54.614346 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:44207
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.254.keytab
--principal=oryx/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:44207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:54.939739 19899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:54.940068 19899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:54.940198 19899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:54.947445 19899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:54.947577 19899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:54.947654 19899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:54.947726 19899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:54.947786 19899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:54.957592 19899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:44207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.254.keytab
--principal=oryx/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:44207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:54.958985 19899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:54.960292 19899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:54.971926 19904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:54.971920 19907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:54.971908 19905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:54.972759 19899 server_base.cc:1034] running on GCE node
I20260927 07:00:54.973551 19899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:54.975046 19899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:54.976264 19899 hybrid_clock.cc:648] HybridClock initialized: now 1790492454976208 us; error 51 us; skew 500 ppm
I20260927 07:00:54.976605 19899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:54 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:54.985994 19899 init.cc:377] Logged in from keytab as oryx/127.13.121.254@KRBTEST.COM (short username oryx)
I20260927 07:00:54.987718 19899 webserver.cc:460] Webserver started at http://127.13.121.254:44105/ using document root <none> and password file <none>
I20260927 07:00:54.988417 19899 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:54.988525 19899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:54.988812 19899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:54.991834 19899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "889e27f4dad54e089c36d203d744b0ab"
format_stamp: "Formatted at 2026-09-27 07:00:54 on dist-test-slave-p6hz"
server_key: "23f2772b2aafcd50b47b3c141fe22451"
server_key_iv: "c12e4c100b6fbc178e335ebdbc47c838"
server_key_version: "encryptionkey@0"
I20260927 07:00:54.992588 19899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "889e27f4dad54e089c36d203d744b0ab"
format_stamp: "Formatted at 2026-09-27 07:00:54 on dist-test-slave-p6hz"
server_key: "23f2772b2aafcd50b47b3c141fe22451"
server_key_iv: "c12e4c100b6fbc178e335ebdbc47c838"
server_key_version: "encryptionkey@0"
I20260927 07:00:54.997871 19899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:00:55.001587 19914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:55.003182 19899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260927 07:00:55.003376 19899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "889e27f4dad54e089c36d203d744b0ab"
format_stamp: "Formatted at 2026-09-27 07:00:54 on dist-test-slave-p6hz"
server_key: "23f2772b2aafcd50b47b3c141fe22451"
server_key_iv: "c12e4c100b6fbc178e335ebdbc47c838"
server_key_version: "encryptionkey@0"
I20260927 07:00:55.003574 19899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:55.015645 19899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:55.019954 19899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:55.020222 19899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:55.037855 19899 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:44207
I20260927 07:00:55.037907 19965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:44207 every 8 connection(s)
I20260927 07:00:55.039250 19899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:00:55.042965 19966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:55.044250 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 19899
I20260927 07:00:55.044462 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:00:55.045269 13799 external_mini_cluster.cc:1428] Setting key 09d85d010085e77a9e51163e35c80e7b
I20260927 07:00:55.057998 19966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: Bootstrap starting.
Sep 27 07:00:55 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:00:55.061887 19966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:55.063225 19966 log.cc:824] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:55.066887 19966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: No bootstrap required, opened a new log
I20260927 07:00:55.070063 19969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:55.047402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53308 (local address 127.13.121.254:44207)
0927 07:00:55.048244 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:55.048256 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:55.048303 (+ 47us) server_negotiation.cc:407] Connection header received
0927 07:00:55.049920 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:55.049946 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:55.050405 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:55.050757 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:55.052595 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:55.054005 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:55.055868 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:55.056645 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:55.056788 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:55.061631 (+ 4843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:55.061641 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:55.061659 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:55.061705 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:55.065748 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:55.066596 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:55.066603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:55.066610 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:55.066794 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:55.067208 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:55.067212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:55.067214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:55.067745 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:55.068080 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:55.068461 (+ 381us) server_negotiation.cc:299] Negotiation successful
0927 07:00:55.068976 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":356,"threads_started":1}
I20260927 07:00:55.075275 19966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } }
I20260927 07:00:55.075659 19966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:55.075750 19966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889e27f4dad54e089c36d203d744b0ab, State: Initialized, Role: FOLLOWER
I20260927 07:00:55.076301 19966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } }
I20260927 07:00:55.076519 19966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:00:55.076637 19966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:00:55.076782 19966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:55.079238 19966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } }
I20260927 07:00:55.079771 19966 leader_election.cc:302] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 889e27f4dad54e089c36d203d744b0ab; no voters:
I20260927 07:00:55.080328 19966 leader_election.cc:288] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:00:55.080470 19971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:00:55.081089 19971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 1 LEADER]: Becoming Leader. State: Replica: 889e27f4dad54e089c36d203d744b0ab, State: Running, Role: LEADER
I20260927 07:00:55.081609 19971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } }
I20260927 07:00:55.082336 19966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:55.084473 19972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } }
I20260927 07:00:55.084467 19973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 889e27f4dad54e089c36d203d744b0ab. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } }
I20260927 07:00:55.085031 19972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:55.085043 19973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:55.085991 19979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:00:55.091759 19979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:00:55.100324 19979 catalog_manager.cc:1347] Generated new cluster ID: da751584e4034fd3b53ed39c1ae55e43
I20260927 07:00:55.100466 19979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:00:55.110678 19979 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:00:55.112109 19979 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:00:55.119709 19979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: Generated new TSK 0
I20260927 07:00:55.120601 19979 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab.
Entry for principal oryx/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab.
I20260927 07:00:55.176575 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:44207
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab
--principal=oryx/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:55.516378 19992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:55.516666 19992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:55.516808 19992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:55.524267 19992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:55.524396 19992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:55.524542 19992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:55.534498 19992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab
--principal=oryx/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:44207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:55.535880 19992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:55.537230 19992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:55.550698 20000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:55.550761 19997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:55.550736 19998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:55.551486 19992 server_base.cc:1034] running on GCE node
I20260927 07:00:55.552139 19992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:55.553160 19992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:55.554394 19992 hybrid_clock.cc:648] HybridClock initialized: now 1790492455554332 us; error 55 us; skew 500 ppm
I20260927 07:00:55.554764 19992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:55 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492455, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:55.558415 19992 init.cc:377] Logged in from keytab as oryx/127.13.121.193@KRBTEST.COM (short username oryx)
I20260927 07:00:55.560112 19992 webserver.cc:460] Webserver started at http://127.13.121.193:45677/ using document root <none> and password file <none>
I20260927 07:00:55.560838 19992 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:55.560971 19992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:55.561259 19992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:55.564604 19992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "6011aebe827d4d8cb9596a63325bc278"
format_stamp: "Formatted at 2026-09-27 07:00:55 on dist-test-slave-p6hz"
server_key: "5151e49c46320b4b714697c55f09ded5"
server_key_iv: "d167a7a092b8641ba36c4ac8a8ff1eaa"
server_key_version: "encryptionkey@0"
I20260927 07:00:55.565889 19992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "6011aebe827d4d8cb9596a63325bc278"
format_stamp: "Formatted at 2026-09-27 07:00:55 on dist-test-slave-p6hz"
server_key: "5151e49c46320b4b714697c55f09ded5"
server_key_iv: "d167a7a092b8641ba36c4ac8a8ff1eaa"
server_key_version: "encryptionkey@0"
I20260927 07:00:55.572748 19992 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 07:00:55.576653 20007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:55.578137 19992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:55.578313 19992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "6011aebe827d4d8cb9596a63325bc278"
format_stamp: "Formatted at 2026-09-27 07:00:55 on dist-test-slave-p6hz"
server_key: "5151e49c46320b4b714697c55f09ded5"
server_key_iv: "d167a7a092b8641ba36c4ac8a8ff1eaa"
server_key_version: "encryptionkey@0"
I20260927 07:00:55.578533 19992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:55.593176 19992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:55.597448 19992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:55.597733 19992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:55.598783 19992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:55.600466 19992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:55.600545 19992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:55.600663 19992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:55.600714 19992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:55.633806 19992 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:43887
I20260927 07:00:55.633898 20119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:43887 every 8 connection(s)
I20260927 07:00:55.635059 19992 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:55.641754 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 19992
I20260927 07:00:55.641932 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:00:55.642532 13799 external_mini_cluster.cc:1428] Setting key 7b7bceb66c1821615b6cbdef7523f4ff
Sep 27 07:00:55 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492455, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:00:55.656904 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:55.637638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:35929 (local address 127.13.121.254:44207)
0927 07:00:55.638160 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:55.638167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:55.639115 (+ 948us) server_negotiation.cc:407] Connection header received
0927 07:00:55.641096 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:55.641101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:55.641170 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:55.641270 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:55.642966 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:55.643888 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:55.645809 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:55.646443 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:55.646598 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:55.650242 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:55.650250 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:55.650255 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:55.650298 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:55.653404 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:55.654280 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:55.654286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:55.654289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:55.654444 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:55.654929 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:55.654934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:55.654937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:55.655312 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:55.655465 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:55.656285 (+ 820us) server_negotiation.cc:299] Negotiation successful
0927 07:00:55.656530 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":228,"threads_started":1}
I20260927 07:00:55.657867 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:55.638186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:35929)
0927 07:00:55.638945 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:55.638993 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:55.640815 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:55.641475 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:55.641485 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:55.642114 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:55.642749 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:55.642771 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:55.644092 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:55.644098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:55.645605 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:55.645620 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:55.646697 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:55.646702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:55.646924 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:55.647659 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:55.647685 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:55.649955 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:55.653609 (+ 3654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:55.653618 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:55.653642 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:55.654148 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:55.654588 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:55.654592 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:55.654595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:55.654808 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:55.655484 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:55.655492 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:55.655865 (+ 373us) client_negotiation.cc:769] Sending connection context
0927 07:00:55.656159 (+ 294us) client_negotiation.cc:240] Negotiation successful
0927 07:00:55.656592 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":302,"threads_started":1}
I20260927 07:00:55.659472 20120 heartbeater.cc:347] Connected to a master server at 127.13.121.254:44207
I20260927 07:00:55.659782 20120 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:55.660530 20120 heartbeater.cc:511] Master 127.13.121.254:44207 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:00:55.662637 19931 ts_manager.cc:194] Registered new tserver with Master: 6011aebe827d4d8cb9596a63325bc278 (127.13.121.193:43887)
I20260927 07:00:55.664278 19931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:35929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab.
Entry for principal oryx/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab.
I20260927 07:00:55.691356 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:44207
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab
--principal=oryx/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:56.015316 20126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:56.015601 20126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:56.015691 20126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:56.022979 20126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:56.023101 20126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:56.023262 20126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:56.033442 20126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab
--principal=oryx/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:44207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:56.034734 20126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:56.036049 20126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:56.050304 20132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:56.050278 20134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:56.050969 20131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:56.051326 20126 server_base.cc:1034] running on GCE node
I20260927 07:00:56.051931 20126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:56.052906 20126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:56.054215 20126 hybrid_clock.cc:648] HybridClock initialized: now 1790492456054150 us; error 54 us; skew 500 ppm
I20260927 07:00:56.054586 20126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:56 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492456, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:56.057732 20126 init.cc:377] Logged in from keytab as oryx/127.13.121.194@KRBTEST.COM (short username oryx)
I20260927 07:00:56.059453 20126 webserver.cc:460] Webserver started at http://127.13.121.194:45463/ using document root <none> and password file <none>
I20260927 07:00:56.060237 20126 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:56.060364 20126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:56.060657 20126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:56.063728 20126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "ef99443da8a04883ab5bd0928f6ed985"
format_stamp: "Formatted at 2026-09-27 07:00:56 on dist-test-slave-p6hz"
server_key: "3c2c50396a9c563b44112d22487d5c1c"
server_key_iv: "a32d2075440f0ebadc8fb35b3983f582"
server_key_version: "encryptionkey@0"
I20260927 07:00:56.064606 20126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "ef99443da8a04883ab5bd0928f6ed985"
format_stamp: "Formatted at 2026-09-27 07:00:56 on dist-test-slave-p6hz"
server_key: "3c2c50396a9c563b44112d22487d5c1c"
server_key_iv: "a32d2075440f0ebadc8fb35b3983f582"
server_key_version: "encryptionkey@0"
I20260927 07:00:56.070052 20126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:00:56.073673 20141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:56.075129 20126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260927 07:00:56.075376 20126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "ef99443da8a04883ab5bd0928f6ed985"
format_stamp: "Formatted at 2026-09-27 07:00:56 on dist-test-slave-p6hz"
server_key: "3c2c50396a9c563b44112d22487d5c1c"
server_key_iv: "a32d2075440f0ebadc8fb35b3983f582"
server_key_version: "encryptionkey@0"
I20260927 07:00:56.075574 20126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:56.091506 20126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:56.096033 20126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:56.096310 20126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:56.097361 20126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:56.100242 20126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:56.100328 20126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:56.100493 20126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:56.100557 20126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:56.134635 20126 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:36965
I20260927 07:00:56.134675 20253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:36965 every 8 connection(s)
I20260927 07:00:56.135941 20126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:56.141512 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 20126
I20260927 07:00:56.141686 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:00:56.142262 13799 external_mini_cluster.cc:1428] Setting key 16067a1340b67c116e3b070862577636
Sep 27 07:00:56 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492456, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:00:56.157531 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:56.138389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:45241 (local address 127.13.121.254:44207)
0927 07:00:56.138588 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:56.138594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:56.139810 (+ 1216us) server_negotiation.cc:407] Connection header received
0927 07:00:56.141622 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:56.141628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:56.141712 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:56.141794 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:56.143827 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:56.144693 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:56.147040 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:56.147506 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:56.147610 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:56.151621 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:56.151626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:56.151629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:56.151665 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:56.154235 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:56.155058 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:56.155062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:56.155065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:56.155249 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:56.155709 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:56.155713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:56.155715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:56.156056 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:56.156193 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:56.156954 (+ 761us) server_negotiation.cc:299] Negotiation successful
0927 07:00:56.157194 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260927 07:00:56.158576 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:56.138854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.194:45241)
0927 07:00:56.139642 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:56.139686 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:56.141328 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:56.142123 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:56.142136 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:56.142907 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:56.143650 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:56.143671 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:56.144864 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:56.144870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:56.146861 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:56.146879 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:56.147734 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:56.147740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:56.148034 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:56.148748 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:56.148777 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:56.151351 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:56.154445 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:56.154454 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:56.154470 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:56.154915 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:56.155367 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:56.155372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:56.155375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:56.155584 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:56.156186 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:56.156195 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:56.156574 (+ 379us) client_negotiation.cc:769] Sending connection context
0927 07:00:56.156815 (+ 241us) client_negotiation.cc:240] Negotiation successful
0927 07:00:56.157376 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":321,"threads_started":1}
I20260927 07:00:56.160333 20254 heartbeater.cc:347] Connected to a master server at 127.13.121.254:44207
I20260927 07:00:56.160699 20254 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:56.161689 20254 heartbeater.cc:511] Master 127.13.121.254:44207 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:00:56.163530 19931 ts_manager.cc:194] Registered new tserver with Master: ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965)
I20260927 07:00:56.164402 19931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:45241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab.
Entry for principal oryx/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab.
I20260927 07:00:56.190980 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:44207
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab
--principal=oryx/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:56.520375 20259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:56.520663 20259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:56.520754 20259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:56.528313 20259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:56.528446 20259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:56.528561 20259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:56.538679 20259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab
--principal=oryx/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:44207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:56.540062 20259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:56.541358 20259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:56.553963 20267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:56.554028 20264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:56.554000 20265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:56.554621 20259 server_base.cc:1034] running on GCE node
I20260927 07:00:56.555269 20259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:56.556201 20259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:56.557428 20259 hybrid_clock.cc:648] HybridClock initialized: now 1790492456557379 us; error 39 us; skew 500 ppm
I20260927 07:00:56.557744 20259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:56 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492456, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:56.560973 20259 init.cc:377] Logged in from keytab as oryx/127.13.121.195@KRBTEST.COM (short username oryx)
I20260927 07:00:56.562479 20259 webserver.cc:460] Webserver started at http://127.13.121.195:41743/ using document root <none> and password file <none>
I20260927 07:00:56.563199 20259 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:56.563309 20259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:56.563578 20259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:56.566525 20259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "6579c7d00e164f199ef48433840423d2"
format_stamp: "Formatted at 2026-09-27 07:00:56 on dist-test-slave-p6hz"
server_key: "a1623b6fbb431c863b9ea10847c36e49"
server_key_iv: "b7e2c2d722e524ee8aa308553d6a9084"
server_key_version: "encryptionkey@0"
I20260927 07:00:56.567366 20259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "6579c7d00e164f199ef48433840423d2"
format_stamp: "Formatted at 2026-09-27 07:00:56 on dist-test-slave-p6hz"
server_key: "a1623b6fbb431c863b9ea10847c36e49"
server_key_iv: "b7e2c2d722e524ee8aa308553d6a9084"
server_key_version: "encryptionkey@0"
I20260927 07:00:56.572614 20259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260927 07:00:56.576188 20274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:56.577715 20259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:00:56.577876 20259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "6579c7d00e164f199ef48433840423d2"
format_stamp: "Formatted at 2026-09-27 07:00:56 on dist-test-slave-p6hz"
server_key: "a1623b6fbb431c863b9ea10847c36e49"
server_key_iv: "b7e2c2d722e524ee8aa308553d6a9084"
server_key_version: "encryptionkey@0"
I20260927 07:00:56.578081 20259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:56.603111 20259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:56.607518 20259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:56.607782 20259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:56.608764 20259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:56.610114 20259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:56.610188 20259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:56.610297 20259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:56.610364 20259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:56.641482 20259 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:36693
I20260927 07:00:56.641530 20386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:36693 every 8 connection(s)
I20260927 07:00:56.642640 20259 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:56.651540 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 20259
I20260927 07:00:56.651681 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:00:56.652094 13799 external_mini_cluster.cc:1428] Setting key 8b481145916936ac11b48b226de94463
Sep 27 07:00:56 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492456, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:00:56.662578 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:56.645143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:60131 (local address 127.13.121.254:44207)
0927 07:00:56.645373 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:56.645380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:56.646640 (+ 1260us) server_negotiation.cc:407] Connection header received
0927 07:00:56.648447 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:56.648453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:56.648576 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:56.648662 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:56.650194 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:56.650940 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:56.652659 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:56.653117 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:56.653240 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:56.657130 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:56.657135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:56.657138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:56.657172 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:56.659639 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:56.660452 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:56.660457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:56.660459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:56.660595 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:56.661052 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:56.661056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:56.661058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:56.661339 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:56.661457 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:56.662162 (+ 705us) server_negotiation.cc:299] Negotiation successful
0927 07:00:56.662326 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260927 07:00:56.663666 20389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:56.645669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.195:60131)
0927 07:00:56.646476 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:56.646517 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:56.648162 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:56.648906 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:56.648918 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:56.649449 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:56.650022 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:56.650040 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:56.651078 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:56.651081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:56.652491 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:56.652504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:56.653383 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:56.653392 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:56.653692 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:56.654459 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:56.654483 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:56.656907 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:56.659782 (+ 2875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:56.659791 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:56.659807 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:56.660275 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:56.660705 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:56.660708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:56.660711 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:56.660911 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:56.661479 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:56.661486 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:56.661825 (+ 339us) client_negotiation.cc:769] Sending connection context
0927 07:00:56.662085 (+ 260us) client_negotiation.cc:240] Negotiation successful
0927 07:00:56.662492 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":310,"threads_started":1}
I20260927 07:00:56.665097 20387 heartbeater.cc:347] Connected to a master server at 127.13.121.254:44207
I20260927 07:00:56.665393 20387 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:56.666322 20387 heartbeater.cc:511] Master 127.13.121.254:44207 requested a full tablet report, sending...
I20260927 07:00:56.667626 20120 heartbeater.cc:503] Master 127.13.121.254:44207 was elected leader, sending a full tablet report...
I20260927 07:00:56.667968 19931 ts_manager.cc:194] Registered new tserver with Master: 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693)
I20260927 07:00:56.668964 19931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:60131
I20260927 07:00:56.680583 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.121.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.253.keytab.
Entry for principal oryx/127.13.121.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.253.keytab.
I20260927 07:00:56.726653 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.253:39055
--webserver_interface=127.13.121.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.253.keytab
--principal=oryx/127.13.121.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:44207,127.13.121.253:39055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:00:57.049060 20392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:57.049381 20392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:57.049525 20392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:57.057096 20392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:57.057240 20392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:57.057322 20392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:57.057394 20392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:57.057461 20392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:57.067731 20392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:44207,127.13.121.253:39055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.253.keytab
--principal=oryx/127.13.121.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.253:39055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.121.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
W20260927 07:00:57.068117 20392 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.121.254:44207,127.13.121.253:39055'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260927 07:00:57.069175 20392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:57.070488 20392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:57.082086 20397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:57.082048 20400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:57.082072 20398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:57.083025 20392 server_base.cc:1034] running on GCE node
I20260927 07:00:57.083957 20392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:57.085403 20392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:57.086654 20392 hybrid_clock.cc:648] HybridClock initialized: now 1790492457086582 us; error 42 us; skew 500 ppm
I20260927 07:00:57.086989 20392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492457, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:57.090159 20392 init.cc:377] Logged in from keytab as oryx/127.13.121.253@KRBTEST.COM (short username oryx)
I20260927 07:00:57.091871 20392 webserver.cc:460] Webserver started at http://127.13.121.253:39011/ using document root <none> and password file <none>
I20260927 07:00:57.092623 20392 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:57.092724 20392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:57.093053 20392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:57.096375 20392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data/instance:
uuid: "4c72c27da588488b9d00ed0d698884bc"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "7ccdd9d3d0aeff57680d6bf91600a817"
server_key_iv: "e092d8d0d4b1e126e39b8ac3a2b68483"
server_key_version: "encryptionkey@0"
I20260927 07:00:57.097188 20392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal/instance:
uuid: "4c72c27da588488b9d00ed0d698884bc"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "7ccdd9d3d0aeff57680d6bf91600a817"
server_key_iv: "e092d8d0d4b1e126e39b8ac3a2b68483"
server_key_version: "encryptionkey@0"
I20260927 07:00:57.102535 20392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:00:57.106418 20407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:57.108233 20392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:00:57.108433 20392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal
uuid: "4c72c27da588488b9d00ed0d698884bc"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "7ccdd9d3d0aeff57680d6bf91600a817"
server_key_iv: "e092d8d0d4b1e126e39b8ac3a2b68483"
server_key_version: "encryptionkey@0"
I20260927 07:00:57.108662 20392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:57.132874 20392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:57.138341 20392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:57.138613 20392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:57.156741 20392 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.253:39055
I20260927 07:00:57.156759 20458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.253:39055 every 8 connection(s)
I20260927 07:00:57.158140 20392 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
I20260927 07:00:57.162235 20459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:57.166363 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 20392
I20260927 07:00:57.167233 20254 heartbeater.cc:503] Master 127.13.121.254:44207 was elected leader, sending a full tablet report...
I20260927 07:00:57.170082 20459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.253@KRBTEST.COM
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492457, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.253@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:00:57.190063 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.174205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.13.121.254:44207)
0927 07:00:57.175303 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.175307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.175322 (+ 15us) server_negotiation.cc:407] Connection header received
0927 07:00:57.175368 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.175371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.175425 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.175521 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.176750 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.177528 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.179193 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.179642 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.179763 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.183739 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.183745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.183749 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.183790 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.187002 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.187803 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.187808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.187812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.187970 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.188351 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.188355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.188358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.188726 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.188871 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.189517 (+ 646us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.189728 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1007,"thread_start_us":914,"threads_started":1}
I20260927 07:00:57.191030 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.174111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:53314)
0927 07:00:57.174726 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.174775 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.174923 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.175606 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.175614 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.176259 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.176575 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.176586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.177654 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.177658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.179029 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.179040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.179862 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.179870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.180056 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.180676 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.180708 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.183547 (+ 2839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.187127 (+ 3580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.187140 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.187153 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.187692 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.188070 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.188075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.188078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.188255 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.188897 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.188906 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.189126 (+ 220us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.189404 (+ 278us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.189937 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":91,"thread_start_us":242,"threads_started":1}
I20260927 07:00:57.191048 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.169029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36536 (local address 127.13.121.253:39055)
0927 07:00:57.170029 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.170042 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.170087 (+ 45us) server_negotiation.cc:407] Connection header received
0927 07:00:57.172020 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.172044 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.172479 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.172796 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.174552 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.176125 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.177521 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.178323 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.178452 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.183342 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.183354 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.183372 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.183411 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.186646 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.187526 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.187532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.187535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.187741 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.188100 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.188103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.188105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.188774 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.189041 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.189391 (+ 350us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.189786 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"spinlock_wait_cycles":16768,"thread_start_us":527,"threads_started":1}
I20260927 07:00:57.193034 20459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492457, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.253@KRBTEST.COM for oryx/127.13.121.253@KRBTEST.COM
I20260927 07:00:57.204883 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.193692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.0.0.1:36550)
0927 07:00:57.193926 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.193945 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.194048 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.194303 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.194308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.194656 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.194781 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.194789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.195745 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.195749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.196596 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.196607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.197228 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.197234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.197410 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.197935 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.197950 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.199943 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.202654 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.202659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.202664 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.203038 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.203403 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.203409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.203412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.203533 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.204000 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.204005 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.204134 (+ 129us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.204244 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.204520 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260927 07:00:57.204921 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.193941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36550 (local address 127.13.121.253:39055)
0927 07:00:57.194081 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.194087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.194102 (+ 15us) server_negotiation.cc:407] Connection header received
0927 07:00:57.194143 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.194147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.194201 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.194276 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.194896 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.195620 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.196719 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.197104 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.197209 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.200115 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.200120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.200123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.200152 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.202515 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.203182 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.203186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.203189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.203298 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.203623 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.203628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.203631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.203898 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.204021 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.204333 (+ 312us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.204654 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":133,"server-negotiator.queue_time_us":57}
I20260927 07:00:57.209787 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Bootstrap starting.
I20260927 07:00:57.212978 20459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:57.214078 20459 log.cc:824] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:57.216995 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: No bootstrap required, opened a new log
I20260927 07:00:57.224046 20459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } }
I20260927 07:00:57.224442 20459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:57.224538 20459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c72c27da588488b9d00ed0d698884bc, State: Initialized, Role: FOLLOWER
I20260927 07:00:57.225164 20459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } }
I20260927 07:00:57.225982 20466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } } }
I20260927 07:00:57.226260 20466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:00:57.226963 20459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: configured and running, proceeding with master startup.
W20260927 07:00:57.235297 20477 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:00:57.235452 20477 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:00:57.237251 20392 master_runner.cc:428] Detected that this master 4c72c27da588488b9d00ed0d698884bc is joining an existing cluster
I20260927 07:00:57.237337 20392 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.121.253:39055
I20260927 07:00:57.257340 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.240955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53320 (local address 127.13.121.254:44207)
0927 07:00:57.241148 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.241155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.241739 (+ 584us) server_negotiation.cc:407] Connection header received
0927 07:00:57.241942 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.241948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.242069 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.242156 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.243897 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.244845 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.246905 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.247546 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.247644 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.250124 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.250130 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.250135 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.250174 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.254070 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.254939 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.254955 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.254959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.255115 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.255613 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.255618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.255621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.255988 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.256106 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.256731 (+ 625us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.256995 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:00:57.258143 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.241804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.0.0.1:36562)
0927 07:00:57.243772 (+ 1968us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.243791 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.243960 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.244281 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.244286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.244879 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.245267 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.245284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.246901 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.246907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.248430 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.248444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.249755 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.249762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.249956 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.250511 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.250533 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.251638 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.255516 (+ 3878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.255520 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.255524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.255996 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.256385 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.256390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.256393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.256550 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.257161 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.257165 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.257298 (+ 133us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.257422 (+ 124us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.257722 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1825,"mutex_wait_us":27,"thread_start_us":1648,"threads_started":1}
I20260927 07:00:57.259202 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.241963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36562 (local address 127.13.121.253:39055)
0927 07:00:57.242116 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.242128 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.243879 (+ 1751us) server_negotiation.cc:407] Connection header received
0927 07:00:57.244075 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.244081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.244148 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.244258 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.245438 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.246670 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.248591 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.249342 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.249446 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.251825 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.251844 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.251849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.251883 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.255379 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.256117 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.256121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.256125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.256262 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.256646 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.256650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.256652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.256960 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.257112 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.258443 (+ 1331us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.258797 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":53}
I20260927 07:00:57.259220 20483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.240985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:53320)
0927 07:00:57.241631 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.241650 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.241782 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.242393 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.242398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.243207 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.243740 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.243755 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.245008 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.245013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.246716 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.246732 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.247963 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.247970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.248144 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.248815 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.248838 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.249898 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.254271 (+ 4373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.254277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.254281 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.254778 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.255319 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.255323 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.255328 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.255487 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.256182 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.256187 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.256389 (+ 202us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.257896 (+ 1507us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.258635 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":1,"spinlock_wait_cycles":49024,"thread_start_us":313,"threads_started":1}
WARNING: no policy specified for oryx/127.13.121.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.252@KRBTEST.COM" created.
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.254@KRBTEST.COM for oryx/127.13.121.253@KRBTEST.COM
I20260927 07:00:57.281951 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.264865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.13.121.253:39055)
0927 07:00:57.265056 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.265063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.267963 (+ 2900us) server_negotiation.cc:407] Connection header received
0927 07:00:57.268356 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.268364 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.268469 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.268555 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.269709 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.270447 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.271928 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.272436 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.272596 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.276091 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.276101 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.276105 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.276150 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.278951 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.279760 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.279764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.279766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.279894 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.280343 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.280347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.280349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.280671 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.280802 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.281333 (+ 531us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.281574 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 07:00:57.281985 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.264839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.0.0.1:36578)
0927 07:00:57.267784 (+ 2945us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.267830 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.268129 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.268639 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.268648 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.269311 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.269532 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.269542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.270617 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.270622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.271779 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.271792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.272632 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.272639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.272829 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.273472 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.273503 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.275839 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.279093 (+ 3254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.279108 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.279117 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.279631 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.280007 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.280013 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.280017 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.280232 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.280790 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.280799 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.281026 (+ 227us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.281209 (+ 183us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.281581 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2706,"thread_start_us":2567,"threads_started":1}
I20260927 07:00:57.284395 19931 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.121.253:39055: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4c72c27da588488b9d00ed0d698884bc"
member_type: NON_VOTER
last_known_addr {
host: "127.13.121.253"
port: 39055
}
attrs {
promote: true
}
}
dest_uuid: "889e27f4dad54e089c36d203d744b0ab"
cas_config_opid_index: -1
I20260927 07:00:57.285470 19931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } }
I20260927 07:00:57.299218 20466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:00:57.299521 20466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } }
I20260927 07:00:57.300787 20466 leader_election.cc:288] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207)
I20260927 07:00:57.301637 19941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c72c27da588488b9d00ed0d698884bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889e27f4dad54e089c36d203d744b0ab" is_pre_election: true
I20260927 07:00:57.302006 20489 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 4c72c27da588488b9d00ed0d698884bc (127.13.121.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } } }
I20260927 07:00:57.303668 20434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.252.keytab.
Entry for principal oryx/127.13.121.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.252.keytab.
I20260927 07:00:57.306106 20489 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.121.253:39055
I20260927 07:00:57.306581 20489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } } }
I20260927 07:00:57.306838 20489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:57.306990 20392 master.cc:537] Master@127.13.121.253:39055 shutting down...
I20260927 07:00:57.307562 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.252:45047
--webserver_interface=127.13.121.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.252.keytab
--principal=oryx/127.13.121.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:44207,127.13.121.253:39055,127.13.121.252:45047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:00:57.311692 20410 leader_election.cc:302] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 4c72c27da588488b9d00ed0d698884bc; no voters: 889e27f4dad54e089c36d203d744b0ab
I20260927 07:00:57.312583 20434 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Refusing update from remote peer 889e27f4dad54e089c36d203d744b0ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260927 07:00:57.312799 20466 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 07:00:57.313892 20490 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 07:00:57.317138 19917 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260927 07:00:57.317534 20392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:57.317982 20392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:57.318171 20392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: stopping tablet replica
I20260927 07:00:57.324802 20392 master.cc:552] Master@127.13.121.253:39055 shutdown complete.
I20260927 07:00:57.324900 20392 master_runner.cc:305] Clearing existing system tablet
I20260927 07:00:57.325309 20392 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:57.325400 20392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:57.334556 20392 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.004s sys 0.006s
W20260927 07:00:57.334602 19917 consensus_peers.cc:487] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab -> Peer 4c72c27da588488b9d00ed0d698884bc (127.13.121.253:39055): Couldn't send request to peer 4c72c27da588488b9d00ed0d698884bc. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260927 07:00:57.336225 20495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:57.336584 20392 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:57.337039 20392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 07:00:57.337183 20392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal
uuid: "4c72c27da588488b9d00ed0d698884bc"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "7ccdd9d3d0aeff57680d6bf91600a817"
server_key_iv: "e092d8d0d4b1e126e39b8ac3a2b68483"
server_key_version: "encryptionkey@0"
I20260927 07:00:57.342350 20392 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:57.349574 20392 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 07:00:57.349895 20392 log.cc:1196] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260927 07:00:57.350378 20392 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Deleting consensus metadata
I20260927 07:00:57.350687 20392 master_runner.cc:315] Copying system tablet from 127.13.121.254:44207
I20260927 07:00:57.353332 20392 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: tablet copy: Beginning tablet copy session from remote peer at address 127.13.121.254:44207
I20260927 07:00:57.369855 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.354285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.13.121.254:44207)
0927 07:00:57.354462 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.354469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.355046 (+ 577us) server_negotiation.cc:407] Connection header received
0927 07:00:57.355338 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.355345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.355435 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.355500 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.356730 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.357873 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.359836 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.360449 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.360561 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.363066 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.363073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.363078 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.363118 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.366413 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.367396 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.367417 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.367421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.367625 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.368147 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.368152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.368155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.368550 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.368692 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.369255 (+ 563us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.369498 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 07:00:57.371289 20501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.354215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:53336)
0927 07:00:57.354933 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.354957 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.355188 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.355715 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.355720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.356237 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.356549 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.356562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.358070 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.358076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.359617 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.359636 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.360838 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.360844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.361040 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.361723 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.361745 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.362831 (+ 1086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.366633 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.366638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.366642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.367183 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.367812 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.367817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.367820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.368005 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.368779 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.368784 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.368968 (+ 184us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.370415 (+ 1447us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.370829 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":439,"threads_started":1}
I20260927 07:00:57.372474 19951 tablet_copy_service.cc:140] P 889e27f4dad54e089c36d203d744b0ab: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4c72c27da588488b9d00ed0d698884bc ({username='oryx', principal='oryx/127.13.121.253@KRBTEST.COM'} at 127.0.0.1:53336)
I20260927 07:00:57.372743 19951 tablet_copy_service.cc:161] P 889e27f4dad54e089c36d203d744b0ab: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4c72c27da588488b9d00ed0d698884bc at {username='oryx', principal='oryx/127.13.121.253@KRBTEST.COM'} at 127.0.0.1:53336: session id = 4c72c27da588488b9d00ed0d698884bc-00000000000000000000000000000000
I20260927 07:00:57.377250 19951 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 07:00:57.379791 20392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 07:00:57.386873 20392 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: tablet copy: Starting download of 0 data blocks...
I20260927 07:00:57.387247 20392 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: tablet copy: Starting download of 1 WAL segments...
I20260927 07:00:57.390209 20392 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 07:00:57.395352 19951 tablet_copy_service.cc:342] P 889e27f4dad54e089c36d203d744b0ab: Request end of tablet copy session 4c72c27da588488b9d00ed0d698884bc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.121.253@KRBTEST.COM'} at 127.0.0.1:53336
I20260927 07:00:57.395725 19951 tablet_copy_service.cc:433] P 889e27f4dad54e089c36d203d744b0ab: ending tablet copy session 4c72c27da588488b9d00ed0d698884bc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4c72c27da588488b9d00ed0d698884bc
I20260927 07:00:57.407605 20392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:57.410444 20502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:57.416142 20503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:57.423229 20392 server_base.cc:1034] running on GCE node
W20260927 07:00:57.423116 20505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:57.423856 20392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:57.424563 20392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:57.425803 20392 hybrid_clock.cc:648] HybridClock initialized: now 1790492457425729 us; error 41 us; skew 500 ppm
I20260927 07:00:57.426035 20392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492457, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:57.428282 20392 init.cc:377] Logged in from keytab as oryx/127.13.121.253@KRBTEST.COM (short username oryx)
I20260927 07:00:57.429935 20392 webserver.cc:460] Webserver started at http://127.13.121.253:35891/ using document root <none> and password file <none>
I20260927 07:00:57.430318 20392 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:57.430414 20392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:57.433547 20392 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260927 07:00:57.435485 20512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:57.435910 20392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260927 07:00:57.436049 20392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal
uuid: "4c72c27da588488b9d00ed0d698884bc"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "7ccdd9d3d0aeff57680d6bf91600a817"
server_key_iv: "e092d8d0d4b1e126e39b8ac3a2b68483"
server_key_version: "encryptionkey@0"
I20260927 07:00:57.436218 20392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:57.471202 20392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:57.475596 20392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:57.503598 20392 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.253:39055
I20260927 07:00:57.503645 20563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.253:39055 every 8 connection(s)
I20260927 07:00:57.504985 20392 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
I20260927 07:00:57.510725 20564 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:00:57.514906 20564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Bootstrap starting.
I20260927 07:00:57.533066 20564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:00:57.533715 20564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Bootstrap complete.
I20260927 07:00:57.534870 20564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } }
I20260927 07:00:57.535175 20564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4c72c27da588488b9d00ed0d698884bc, State: Initialized, Role: LEARNER
I20260927 07:00:57.535475 20564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } }
I20260927 07:00:57.536339 20567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } } }
I20260927 07:00:57.536731 20564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:57.536775 20567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: This master's current role is: LEARNER
I20260927 07:00:57.551066 20578 catalog_manager.cc:1259] Loaded cluster ID: da751584e4034fd3b53ed39c1ae55e43
I20260927 07:00:57.551192 20578 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: loading cluster ID for follower catalog manager: success
I20260927 07:00:57.553776 20578 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: acquiring CA information for follower catalog manager: success
I20260927 07:00:57.555774 20578 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:00:57.671401 20387 heartbeater.cc:503] Master 127.13.121.254:44207 was elected leader, sending a full tablet report...
W20260927 07:00:57.740772 20492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:57.741084 20492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:57.741211 20492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:57.748664 20492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:00:57.748796 20492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:57.748872 20492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:57.748951 20492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:00:57.749011 20492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:00:57.759171 20492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:44207,127.13.121.253:39055,127.13.121.252:45047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.252.keytab
--principal=oryx/127.13.121.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.252:45047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.121.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:57.760682 20492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:57.762130 20492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 07:00:57.768273 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.761798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.0.0.1:41518)
0927 07:00:57.761993 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.762015 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.762145 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.762971 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.762976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.763006 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:57.763229 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.763240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.764557 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.764563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.765790 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.765805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.767475 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.767482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.767692 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.767756 (+ 64us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.767862 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.768007 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260927 07:00:57.768859 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.761955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.13.121.253:39055)
0927 07:00:57.762630 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.762634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.762653 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:00:57.762711 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.762714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.762781 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.762932 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:57.763388 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.764404 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.765978 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.767233 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.767460 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.767651 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.767929 (+ 278us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.768083 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":442,"threads_started":1}
I20260927 07:00:57.770020 20539 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260927 07:00:57.770568 20567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 889e27f4dad54e089c36d203d744b0ab. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } } }
I20260927 07:00:57.770802 20567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: This master's current role is: LEARNER
W20260927 07:00:57.774757 20587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:57.774834 20585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:57.775005 20584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:57.776021 20492 server_base.cc:1034] running on GCE node
I20260927 07:00:57.776793 20492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:57.778165 20492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:57.779381 20492 hybrid_clock.cc:648] HybridClock initialized: now 1790492457779308 us; error 54 us; skew 500 ppm
I20260927 07:00:57.779721 20492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492457, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:57.782965 20492 init.cc:377] Logged in from keytab as oryx/127.13.121.252@KRBTEST.COM (short username oryx)
I20260927 07:00:57.784466 20492 webserver.cc:460] Webserver started at http://127.13.121.252:34283/ using document root <none> and password file <none>
I20260927 07:00:57.785136 20492 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:57.785252 20492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:57.785532 20492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:00:57.788658 20492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data/instance:
uuid: "24cb125cd3904d5e89fd88c34e951906"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "d62e9c1635ac28c770ff73d28c469b0a"
server_key_iv: "e98d7bcd57df600ee4af220ab8bfb112"
server_key_version: "encryptionkey@0"
I20260927 07:00:57.789407 20492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal/instance:
uuid: "24cb125cd3904d5e89fd88c34e951906"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "d62e9c1635ac28c770ff73d28c469b0a"
server_key_iv: "e98d7bcd57df600ee4af220ab8bfb112"
server_key_version: "encryptionkey@0"
I20260927 07:00:57.794483 20492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 07:00:57.798388 20594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:57.799854 20492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:00:57.800036 20492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal
uuid: "24cb125cd3904d5e89fd88c34e951906"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "d62e9c1635ac28c770ff73d28c469b0a"
server_key_iv: "e98d7bcd57df600ee4af220ab8bfb112"
server_key_version: "encryptionkey@0"
I20260927 07:00:57.800226 20492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:57.828781 20492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:57.832856 20492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:57.833132 20492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:57.850559 20492 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.252:45047
I20260927 07:00:57.850564 20645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.252:45047 every 8 connection(s)
I20260927 07:00:57.852075 20492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
I20260927 07:00:57.856021 20646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:00:57.857771 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 20492
I20260927 07:00:57.865447 20646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.252@KRBTEST.COM
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492457, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.252@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:00:57.883412 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.860005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.13.121.252:45047)
0927 07:00:57.860881 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.860898 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.860956 (+ 58us) server_negotiation.cc:407] Connection header received
0927 07:00:57.862756 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.862797 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.863455 (+ 658us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.863902 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.865239 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.867162 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.868647 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.869684 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.869856 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.874241 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.874253 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.874279 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.874326 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.877620 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.878472 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.878479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.878487 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.878691 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.879199 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.879205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.879208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.879873 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.880617 (+ 744us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.881482 (+ 865us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.882009 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":357,"threads_started":1}
I20260927 07:00:57.884413 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.868556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.13.121.254:44207)
0927 07:00:57.868751 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.868758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.869465 (+ 707us) server_negotiation.cc:407] Connection header received
0927 07:00:57.869631 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.869635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.869697 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.869790 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.870919 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.871633 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.872978 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.873577 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.873715 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.877456 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.877461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.877465 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.877502 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.880941 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.882134 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.882141 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.882145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.882303 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.882739 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.882743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.882746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.883200 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.883339 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.883844 (+ 505us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.884088 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 07:00:57.884496 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.868472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35040)
0927 07:00:57.869319 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.869363 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.869519 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.869872 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.869879 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.870525 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.870755 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.870768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.871769 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.871773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.872827 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.872839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.873819 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.873825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.874002 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.874690 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.874710 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.877200 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.881136 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.881149 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.881158 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.881824 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.882419 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.882424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.882428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.882602 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.883320 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.883328 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.883516 (+ 188us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.883737 (+ 221us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.884088 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":399,"threads_started":1}
I20260927 07:00:57.887107 20646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492457, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.252@KRBTEST.COM for oryx/127.13.121.253@KRBTEST.COM
I20260927 07:00:57.900017 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.887847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.0.0.1:41532)
0927 07:00:57.888071 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.888087 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.888170 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.888456 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.888459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.888770 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.888920 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.888928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.889921 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.889925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.890855 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.890877 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.891660 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.891666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.891819 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.892416 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.892436 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.894474 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.897479 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.897485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.897487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.897990 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.898433 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.898437 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.898439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.898619 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.899238 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.899241 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.899376 (+ 135us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.899502 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.899766 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260927 07:00:57.900074 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.888010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41532 (local address 127.13.121.253:39055)
0927 07:00:57.888151 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.888156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.888172 (+ 16us) server_negotiation.cc:407] Connection header received
0927 07:00:57.888273 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.888277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.888352 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.888450 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.889063 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.889759 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.891008 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.891517 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.891625 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.894696 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.894702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.894707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.894749 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.897267 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.898134 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.898139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.898141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.898290 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.898742 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.898746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.898748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.899076 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.899231 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.899620 (+ 389us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.899819 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 07:00:57.901042 20646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492457, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.252@KRBTEST.COM for oryx/127.13.121.252@KRBTEST.COM
I20260927 07:00:57.913657 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.901873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:45047 (local address 127.0.0.1:38608)
0927 07:00:57.902062 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.902078 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.902168 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.902556 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.902559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.902903 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.903083 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.903090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.904075 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.904079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.905076 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.905088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.905795 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.905800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.905989 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.906599 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.906623 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.908783 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.911438 (+ 2655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.911443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.911448 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.911774 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.912074 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.912079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.912083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.912215 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.912696 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.912701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.912861 (+ 160us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.912972 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.913228 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 07:00:57.913666 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.901916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.13.121.252:45047)
0927 07:00:57.902060 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.902064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.902129 (+ 65us) server_negotiation.cc:407] Connection header received
0927 07:00:57.902297 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.902301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.902402 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.902487 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.903240 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.903918 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.905239 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.905656 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.905767 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.908982 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.908987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.908990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.909018 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.911311 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.911874 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.911879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.911883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.911984 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.912307 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.912312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.912316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.912594 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.912716 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.913069 (+ 353us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.913377 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":49}
I20260927 07:00:57.918449 20646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: Bootstrap starting.
I20260927 07:00:57.921665 20646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: Neither blocks nor log segments found. Creating new log.
I20260927 07:00:57.922693 20646 log.cc:824] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: Log is configured to *not* fsync() on all Append() calls
I20260927 07:00:57.925344 20646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: No bootstrap required, opened a new log
I20260927 07:00:57.932330 20646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } }
I20260927 07:00:57.932690 20646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:00:57.932804 20646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24cb125cd3904d5e89fd88c34e951906, State: Initialized, Role: FOLLOWER
I20260927 07:00:57.933348 20646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } }
I20260927 07:00:57.934217 20652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } } }
I20260927 07:00:57.934520 20652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:00:57.935235 20646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 07:00:57.944344 20663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:00:57.944501 20663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:00:57.948006 20492 master_runner.cc:428] Detected that this master 24cb125cd3904d5e89fd88c34e951906 is joining an existing cluster
I20260927 07:00:57.948134 20492 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.121.252:45047
I20260927 07:00:57.972487 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.954466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.13.121.254:44207)
0927 07:00:57.954659 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.954666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.954855 (+ 189us) server_negotiation.cc:407] Connection header received
0927 07:00:57.955171 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.955177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.955265 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.955345 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.957366 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.958662 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.961018 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.961745 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.961854 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.965029 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.965038 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.965044 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.965091 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.968446 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.969966 (+ 1520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.969973 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.969976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.970167 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.970620 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.970626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.970629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.970969 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.971096 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.971830 (+ 734us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.972113 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 07:00:57.973641 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.954109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35054)
0927 07:00:57.954739 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.954761 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.954985 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.955874 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.955880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.956700 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.957141 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.957156 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.958912 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.958919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.960783 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.960806 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.962237 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.962245 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.962463 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.963397 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.963425 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.964740 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.969209 (+ 4469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.969214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.969218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.969782 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.970307 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.970311 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.970315 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.970497 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.971226 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.971234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.971463 (+ 229us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.972762 (+ 1299us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.973130 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":559,"threads_started":1}
I20260927 07:00:57.982703 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.954667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38610 (local address 127.13.121.252:45047)
0927 07:00:57.954868 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.954874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.965652 (+ 10778us) server_negotiation.cc:407] Connection header received
0927 07:00:57.966048 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.966054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.966117 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.966177 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.968256 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.969539 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.971552 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.972314 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.972457 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.975674 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.975680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.975689 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.975727 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.979533 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.980325 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.980331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.980335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.980480 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.980904 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.980910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.980914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.981299 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.981463 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.981953 (+ 490us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.982325 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":69}
I20260927 07:00:57.984426 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.955515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.13.121.253:39055)
0927 07:00:57.955686 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.955692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.968100 (+ 12408us) server_negotiation.cc:407] Connection header received
0927 07:00:57.968282 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.968289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.968384 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.968456 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.970209 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.971228 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.973922 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.974485 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.974579 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.978118 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.978126 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.978131 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.978172 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.981390 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.982120 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.982126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.982129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.982295 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.982752 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.982757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.982760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.983120 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:57.983261 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:57.983868 (+ 607us) server_negotiation.cc:299] Negotiation successful
0927 07:00:57.984092 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 07:00:57.985487 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.955167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.0.0.1:41536)
0927 07:00:57.967959 (+ 12792us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.967980 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.968104 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.968672 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.968676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.969616 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.970029 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.970042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.971390 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.971395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.973750 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.973764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.974935 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.974941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.975059 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.976704 (+ 1645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.976731 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.977875 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.981527 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.981532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.981536 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.981989 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.982441 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.982446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.982449 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.982621 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.983354 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.983359 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.983573 (+ 214us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.984682 (+ 1109us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.985010 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12638,"mutex_wait_us":4,"thread_start_us":10503,"threads_started":1}
I20260927 07:00:57.986770 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.954230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:45047 (local address 127.0.0.1:38610)
0927 07:00:57.965560 (+ 11330us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.965581 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.965922 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.966379 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.966383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.967039 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.967428 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.967441 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.969744 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.969749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.971384 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.971398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.972529 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.972539 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.973021 (+ 482us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.974030 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.974052 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.975241 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.979672 (+ 4431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.979679 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.979683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.980196 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.980606 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.980610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.980614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.980778 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.981527 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.981532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.981711 (+ 179us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.986034 (+ 4323us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.986328 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11145,"mutex_wait_us":1,"thread_start_us":13401,"threads_started":1}
I20260927 07:00:57.989346 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 19992
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.254@KRBTEST.COM for oryx/127.13.121.252@KRBTEST.COM
Sep 27 07:00:57 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.193@KRBTEST.COM
I20260927 07:00:57.999692 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.979513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:45047 (local address 127.0.0.1:38622)
0927 07:00:57.979674 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:57.979692 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:57.979791 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:57.983893 (+ 4102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:57.983898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:57.984357 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:57.984546 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.984559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.986586 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.986593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.987851 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:57.987868 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.989051 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.989058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:57.989250 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.989834 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:57.989867 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:57.992620 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:57.996622 (+ 4002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.996628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.996632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.997146 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.997623 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:57.997628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:57.997632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:57.997794 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:57.998621 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:57.998626 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:57.998791 (+ 165us) client_negotiation.cc:769] Sending connection context
0927 07:00:57.998919 (+ 128us) client_negotiation.cc:240] Negotiation successful
0927 07:00:57.999294 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260927 07:00:58.000839 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.979883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.13.121.252:45047)
0927 07:00:57.983473 (+ 3590us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.983482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.983501 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:00:57.983601 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.983607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.983711 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.984924 (+ 1213us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.985107 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.986421 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.988022 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.988885 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.989010 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:57.992851 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:57.992858 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:57.992863 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:57.992901 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:57.996451 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.997315 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.997320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.997324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.997489 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:57.998130 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:57.998135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:57.998139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:57.998485 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:58.000117 (+ 1632us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:58.000245 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 07:00:58.000470 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3495,"thread_start_us":6972,"threads_started":1}
I20260927 07:00:58.002297 19931 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.121.252:45047: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "24cb125cd3904d5e89fd88c34e951906"
member_type: NON_VOTER
last_known_addr {
host: "127.13.121.252"
port: 45047
}
attrs {
promote: true
}
}
dest_uuid: "889e27f4dad54e089c36d203d744b0ab"
cas_config_opid_index: 5
I20260927 07:00:58.003432 19931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } }
I20260927 07:00:58.005259 20489 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.006234 20621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:00:58.008880 20489 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.121.252:45047
I20260927 07:00:58.010109 20539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 LEARNER]: Refusing update from remote peer 889e27f4dad54e089c36d203d744b0ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 07:00:58.014520 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:57.990895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.13.121.193:43887)
0927 07:00:57.991511 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:57.991519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:57.991544 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:00:57.991597 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:57.991603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:57.991818 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:57.992031 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:57.993183 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.994326 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.995651 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:57.996221 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:57.996344 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.000500 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:58.000514 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:58.000530 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:58.000564 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:58.004006 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.004955 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.010379 (+ 5424us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.010391 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.010646 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.011355 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.011362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.011366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.011900 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:58.013524 (+ 1624us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:58.013724 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 07:00:58.014047 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":310,"threads_started":1}
I20260927 07:00:58.016523 20115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:58.017040 20675 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260927 07:00:58.019665 20539 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.021796 20674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.022156 20674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:58.023943 20567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: NON_VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: true } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.024252 20567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: This master's current role is: LEARNER
I20260927 07:00:58.025327 20675 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: attempting to promote NON_VOTER 4c72c27da588488b9d00ed0d698884bc to VOTER
I20260927 07:00:58.026028 20675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } }
I20260927 07:00:58.034150 20539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 LEARNER]: Refusing update from remote peer 889e27f4dad54e089c36d203d744b0ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260927 07:00:58.036463 20489 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260927 07:00:58.039528 20492 master.cc:537] Master@127.13.121.252:45047 shutting down...
I20260927 07:00:58.039641 20621 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Refusing update from remote peer 889e27f4dad54e089c36d203d744b0ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 07:00:58.041080 20679 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 4c72c27da588488b9d00ed0d698884bc (127.13.121.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.043479 20620 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Refusing update from remote peer 889e27f4dad54e089c36d203d744b0ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260927 07:00:58.043809 20538 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 4c72c27da588488b9d00ed0d698884bc (127.13.121.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.049073 20489 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:00:58.051470 20567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.051739 20567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:00:58.051852 20489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.052186 20489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:58.052886 20489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.053158 20489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
W20260927 07:00:58.060344 19916 consensus_peers.cc:487] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab -> Peer 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252:45047): Couldn't send request to peer 24cb125cd3904d5e89fd88c34e951906. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 07:00:58.060961 20492 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:00:58.061444 20492 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:00:58.061655 20492 tablet_replica.cc:330] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: stopping tablet replica
I20260927 07:00:58.068729 20492 master.cc:552] Master@127.13.121.252:45047 shutdown complete.
I20260927 07:00:58.068907 20492 master_runner.cc:305] Clearing existing system tablet
I20260927 07:00:58.069348 20492 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:58.069537 20492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:58.072793 20492 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260927 07:00:58.074568 20683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:58.074947 20492 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:58.075407 20492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260927 07:00:58.075551 20492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal
uuid: "24cb125cd3904d5e89fd88c34e951906"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "d62e9c1635ac28c770ff73d28c469b0a"
server_key_iv: "e98d7bcd57df600ee4af220ab8bfb112"
server_key_version: "encryptionkey@0"
I20260927 07:00:58.080732 20492 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:00:58.088438 20492 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 07:00:58.088744 20492 log.cc:1196] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260927 07:00:58.089252 20492 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: Deleting consensus metadata
I20260927 07:00:58.089560 20492 master_runner.cc:315] Copying system tablet from 127.13.121.254:44207
I20260927 07:00:58.094904 20492 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: tablet copy: Beginning tablet copy session from remote peer at address 127.13.121.254:44207
I20260927 07:00:58.121320 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.101597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.13.121.254:44207)
0927 07:00:58.106032 (+ 4435us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:58.106039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:58.106637 (+ 598us) server_negotiation.cc:407] Connection header received
0927 07:00:58.106864 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:58.106870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:58.106953 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:58.107059 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:58.108122 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.109244 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.111071 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.111733 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.111854 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.114276 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:58.114283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:58.114288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:58.114339 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:58.118205 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.119126 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.119167 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.119172 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.119333 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.119746 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.119751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.119754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.120119 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:58.120266 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:58.120721 (+ 455us) server_negotiation.cc:299] Negotiation successful
0927 07:00:58.120961 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4248}
I20260927 07:00:58.121423 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.099697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35068)
0927 07:00:58.106495 (+ 6798us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:58.106517 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:58.106693 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:58.107109 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:58.107114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:58.107702 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:58.107974 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.107988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.109420 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.109425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.110903 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.110918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.111895 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.111906 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.112100 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.112801 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:58.112830 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:58.114029 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:58.118404 (+ 4375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:58.118411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:58.118416 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:58.118983 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:58.119448 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:58.119453 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:58.119457 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:58.119638 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:58.120248 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:58.120253 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:58.120447 (+ 194us) client_negotiation.cc:769] Sending connection context
0927 07:00:58.120609 (+ 162us) client_negotiation.cc:240] Negotiation successful
0927 07:00:58.120940 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6659,"thread_start_us":6482,"threads_started":1}
I20260927 07:00:58.122309 19951 tablet_copy_service.cc:140] P 889e27f4dad54e089c36d203d744b0ab: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 24cb125cd3904d5e89fd88c34e951906 ({username='oryx', principal='oryx/127.13.121.252@KRBTEST.COM'} at 127.0.0.1:35068)
I20260927 07:00:58.122493 19951 tablet_copy_service.cc:161] P 889e27f4dad54e089c36d203d744b0ab: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 24cb125cd3904d5e89fd88c34e951906 at {username='oryx', principal='oryx/127.13.121.252@KRBTEST.COM'} at 127.0.0.1:35068: session id = 24cb125cd3904d5e89fd88c34e951906-00000000000000000000000000000000
I20260927 07:00:58.126281 19951 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 07:00:58.128320 20492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 07:00:58.135470 20492 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: tablet copy: Starting download of 0 data blocks...
I20260927 07:00:58.135792 20492 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: tablet copy: Starting download of 1 WAL segments...
I20260927 07:00:58.139207 20492 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 07:00:58.144722 19951 tablet_copy_service.cc:342] P 889e27f4dad54e089c36d203d744b0ab: Request end of tablet copy session 24cb125cd3904d5e89fd88c34e951906-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.121.252@KRBTEST.COM'} at 127.0.0.1:35068
I20260927 07:00:58.144904 19951 tablet_copy_service.cc:433] P 889e27f4dad54e089c36d203d744b0ab: ending tablet copy session 24cb125cd3904d5e89fd88c34e951906-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 24cb125cd3904d5e89fd88c34e951906
I20260927 07:00:58.157544 20492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:58.160686 20690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:58.164791 20691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:58.176905 20693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:58.178463 20492 server_base.cc:1034] running on GCE node
I20260927 07:00:58.178861 20492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:58.179692 20492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:58.180589 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 19992
I20260927 07:00:58.181649 20492 hybrid_clock.cc:648] HybridClock initialized: now 1790492458181596 us; error 40 us; skew 500 ppm
I20260927 07:00:58.181994 20492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:58 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492458, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:58.184254 20492 init.cc:377] Logged in from keytab as oryx/127.13.121.252@KRBTEST.COM (short username oryx)
I20260927 07:00:58.192240 20492 webserver.cc:460] Webserver started at http://127.13.121.252:39663/ using document root <none> and password file <none>
I20260927 07:00:58.192651 20492 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:58.192756 20492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:58.195740 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:43887
--local_ip_for_outbound_sockets=127.13.121.193
--tserver_master_addrs=127.13.121.254:44207
--webserver_port=45677
--webserver_interface=127.13.121.193
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab
--principal=oryx/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.121.254:44207,127.13.121.253:39055
--tserver_master_addrs=127.13.121.254:44207,127.13.121.253:39055,127.13.121.252:45047 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:00:58.195837 20492 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260927 07:00:58.197708 20700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:58.198158 20492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260927 07:00:58.198314 20492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal
uuid: "24cb125cd3904d5e89fd88c34e951906"
format_stamp: "Formatted at 2026-09-27 07:00:57 on dist-test-slave-p6hz"
server_key: "d62e9c1635ac28c770ff73d28c469b0a"
server_key_iv: "e98d7bcd57df600ee4af220ab8bfb112"
server_key_version: "encryptionkey@0"
I20260927 07:00:58.198546 20492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:58.254339 20492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:58.258504 20492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:58.312795 20492 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.252:45047
I20260927 07:00:58.312848 20752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.252:45047 every 8 connection(s)
I20260927 07:00:58.313832 20492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
I20260927 07:00:58.319339 20753 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:00:58.323235 20753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: Bootstrap starting.
I20260927 07:00:58.343037 20753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:00:58.343556 20753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: Bootstrap complete.
I20260927 07:00:58.344674 20753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } }
I20260927 07:00:58.344990 20753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24cb125cd3904d5e89fd88c34e951906, State: Initialized, Role: LEARNER
I20260927 07:00:58.345258 20753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } }
I20260927 07:00:58.346138 20756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.346484 20753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:00:58.346702 20756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: This master's current role is: LEARNER
I20260927 07:00:58.356036 20767 catalog_manager.cc:1259] Loaded cluster ID: da751584e4034fd3b53ed39c1ae55e43
I20260927 07:00:58.356161 20767 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: loading cluster ID for follower catalog manager: success
I20260927 07:00:58.359757 20767 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: acquiring CA information for follower catalog manager: success
I20260927 07:00:58.362435 20767 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 07:00:58.558874 20703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:58.559212 20703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:58.559351 20703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:58.567078 20703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:58.567262 20703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:58.567431 20703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:00:58.578284 20703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab
--principal=oryx/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:43887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=45677
--tserver_master_addrs=127.13.121.254:44207,127.13.121.253:39055,127.13.121.252:45047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:58.579694 20703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:58.581008 20703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:58.595000 20773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:58.595057 20774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:58.594969 20776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:58.595759 20703 server_base.cc:1034] running on GCE node
I20260927 07:00:58.596341 20703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:58.597273 20703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:58.598498 20703 hybrid_clock.cc:648] HybridClock initialized: now 1790492458598441 us; error 38 us; skew 500 ppm
I20260927 07:00:58.598834 20703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:58 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492458, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:58.602180 20703 init.cc:377] Logged in from keytab as oryx/127.13.121.193@KRBTEST.COM (short username oryx)
I20260927 07:00:58.603732 20703 webserver.cc:460] Webserver started at http://127.13.121.193:45677/ using document root <none> and password file <none>
I20260927 07:00:58.604427 20703 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:58.604559 20703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:58.610069 20703 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 07:00:58.613044 20783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:58.614396 20703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:00:58.614583 20703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "6011aebe827d4d8cb9596a63325bc278"
format_stamp: "Formatted at 2026-09-27 07:00:55 on dist-test-slave-p6hz"
server_key: "5151e49c46320b4b714697c55f09ded5"
server_key_iv: "d167a7a092b8641ba36c4ac8a8ff1eaa"
server_key_version: "encryptionkey@0"
I20260927 07:00:58.615600 20703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:58.630497 20703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:58.634886 20703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:58.635128 20703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:58.636138 20703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:58.637719 20703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:58.637806 20703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:58.637904 20703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:58.637979 20703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:58.662209 20813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.648335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:45047 (local address 127.0.0.1:38636)
0927 07:00:58.656499 (+ 8164us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:58.656521 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:58.656704 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:58.657178 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:58.657183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:58.657216 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:00:58.657420 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.657433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.659035 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.659042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.660273 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.660288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.661393 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.661399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.661598 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.661633 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:00:58.661721 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 07:00:58.661875 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8041,"thread_start_us":7948,"threads_started":1}
I20260927 07:00:58.662207 20815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.648458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.13.121.252:45047)
0927 07:00:58.655713 (+ 7255us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:58.655721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:58.656659 (+ 938us) server_negotiation.cc:407] Connection header received
0927 07:00:58.656859 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:58.656866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:58.656982 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:58.657133 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:00:58.657575 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.658889 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.660437 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.661212 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.661376 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.661578 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:58.661753 (+ 175us) server_negotiation.cc:299] Negotiation successful
0927 07:00:58.661883 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7078,"thread_start_us":6931,"threads_started":1}
I20260927 07:00:58.663448 20728 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260927 07:00:58.664049 20756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 889e27f4dad54e089c36d203d744b0ab. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:58.664324 20756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: This master's current role is: LEARNER
I20260927 07:00:58.681155 20703 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:43887
I20260927 07:00:58.681221 20897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:43887 every 8 connection(s)
I20260927 07:00:58.682699 20703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:00:58.683518 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 20703
I20260927 07:00:58.683844 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 20126
Sep 27 07:00:58 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.194@KRBTEST.COM
Sep 27 07:00:58 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492458, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.252@KRBTEST.COM
Sep 27 07:00:58 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492458, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.253@KRBTEST.COM
Sep 27 07:00:58 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492458, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:00:58.719748 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.685065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.13.121.194:36965)
0927 07:00:58.685546 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:58.685555 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:58.695830 (+ 10275us) server_negotiation.cc:407] Connection header received
0927 07:00:58.696357 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:58.696369 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:58.696689 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:58.696829 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:58.699684 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.701637 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.703873 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.704755 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.704861 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.711519 (+ 6658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:58.711536 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:58.711552 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:58.711602 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:58.715604 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.716953 (+ 1349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.716964 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.716967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.717137 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.717601 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.717606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.717608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.718030 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:58.718299 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:58.718909 (+ 610us) server_negotiation.cc:299] Negotiation successful
0927 07:00:58.719309 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":689,"threads_started":1}
I20260927 07:00:58.721248 20249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:58.728724 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.691958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:45753 (local address 127.13.121.254:44207)
0927 07:00:58.708686 (+ 16728us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:58.708693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:58.709653 (+ 960us) server_negotiation.cc:407] Connection header received
0927 07:00:58.709722 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:58.709728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:58.709806 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:58.709907 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:58.711063 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.712158 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.715240 (+ 3082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.715863 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.715991 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.721714 (+ 5723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:58.721722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:58.721727 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:58.721770 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:58.725448 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.726316 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.726321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.726324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.726460 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.727256 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.727261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.727265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.727616 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:58.727754 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:58.728095 (+ 341us) server_negotiation.cc:299] Negotiation successful
0927 07:00:58.728341 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16584,"thread_start_us":16736,"threads_started":1}
I20260927 07:00:58.729432 20815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.685691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:56869 (local address 127.13.121.252:45047)
0927 07:00:58.685852 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:58.685857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:58.704501 (+ 18644us) server_negotiation.cc:407] Connection header received
0927 07:00:58.704963 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:58.704970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:58.705049 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:58.705114 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:58.708265 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.709358 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.712110 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.712764 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.712872 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.717785 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:58.717791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:58.717796 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:58.717848 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:58.720920 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.725632 (+ 4712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.725638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.725641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.725789 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.726310 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.726315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.726318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.726669 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:58.726795 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:58.728886 (+ 2091us) server_negotiation.cc:299] Negotiation successful
0927 07:00:58.729132 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:00:58.729526 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.691860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:45753)
0927 07:00:58.709174 (+ 17314us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:58.709196 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:58.709328 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:58.710114 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:58.710120 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:58.710589 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:58.710902 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.710911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.713115 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.713119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.714870 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.714884 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.716241 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.716247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.716422 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.716961 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:58.716988 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:58.721112 (+ 4124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:58.725574 (+ 4462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:58.725579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:58.725582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:58.726101 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:58.726855 (+ 754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:58.726859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:58.726863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:58.727005 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:58.727728 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:58.727732 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:58.727860 (+ 128us) client_negotiation.cc:769] Sending connection context
0927 07:00:58.727991 (+ 131us) client_negotiation.cc:240] Negotiation successful
0927 07:00:58.728219 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17199,"mutex_wait_us":71,"thread_start_us":12661,"threads_started":1}
I20260927 07:00:58.730216 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.686304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.13.121.193:40043)
0927 07:00:58.687351 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:58.687412 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:58.690011 (+ 2599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:58.711079 (+ 21068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:58.711084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:58.711548 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:58.711820 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.711832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.713103 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.713109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.714870 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.714884 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.715890 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.715896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.716070 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.716650 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:58.716670 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:58.720397 (+ 3727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:58.724979 (+ 4582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:58.724985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:58.724989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:58.725457 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:58.725890 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:58.725895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:58.725899 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:58.726208 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:58.726839 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:58.726844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:58.727319 (+ 475us) client_negotiation.cc:769] Sending connection context
0927 07:00:58.727563 (+ 244us) client_negotiation.cc:240] Negotiation successful
0927 07:00:58.728110 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":114,"thread_start_us":3312,"threads_started":1}
I20260927 07:00:58.730494 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.686913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:40043 (local address 127.13.121.253:39055)
0927 07:00:58.709714 (+ 22801us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:58.709720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:58.709741 (+ 21us) server_negotiation.cc:407] Connection header received
0927 07:00:58.709789 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:58.709793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:58.709903 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:58.711966 (+ 2063us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:58.712123 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.712971 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.715055 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.715680 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.716515 (+ 835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.720607 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:58.720614 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:58.720619 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:58.720661 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:58.724823 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.725604 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.725610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.725613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.725762 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:58.726362 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:58.726367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:58.726371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:58.726706 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:58.727017 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:58.727672 (+ 655us) server_negotiation.cc:299] Negotiation successful
0927 07:00:58.730146 (+ 2474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22694,"thread_start_us":22609,"threads_started":1}
I20260927 07:00:58.731283 20900 heartbeater.cc:347] Connected to a master server at 127.13.121.253:39055
I20260927 07:00:58.731578 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:58.687283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:45047 (local address 127.13.121.193:56869)
0927 07:00:58.704396 (+ 17113us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:58.704418 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:58.704797 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:58.705636 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:58.705650 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:58.706922 (+ 1272us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:58.708050 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.708078 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.709571 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.709577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.711942 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:58.711956 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:58.713235 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:58.713246 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:58.713562 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:58.714254 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:58.714284 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:58.717543 (+ 3259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:58.721083 (+ 3540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:58.721092 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:58.721108 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:58.723993 (+ 2885us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:58.725952 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:58.725957 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:58.725960 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:58.726191 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:58.726790 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:58.726798 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:58.728579 (+ 1781us) client_negotiation.cc:769] Sending connection context
0927 07:00:58.729607 (+ 1028us) client_negotiation.cc:240] Negotiation successful
0927 07:00:58.729883 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16967,"mutex_wait_us":61,"thread_start_us":21625,"threads_started":1}
I20260927 07:00:58.731621 20900 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:58.732398 20900 heartbeater.cc:511] Master 127.13.121.253:39055 requested a full tablet report, sending...
I20260927 07:00:58.731258 20899 heartbeater.cc:347] Connected to a master server at 127.13.121.254:44207
I20260927 07:00:58.733129 20899 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:58.733525 20899 heartbeater.cc:511] Master 127.13.121.254:44207 requested a full tablet report, sending...
I20260927 07:00:58.734527 19931 ts_manager.cc:194] Re-registered known tserver with Master: 6011aebe827d4d8cb9596a63325bc278 (127.13.121.193:43887)
I20260927 07:00:58.734706 20529 ts_manager.cc:194] Registered new tserver with Master: 6011aebe827d4d8cb9596a63325bc278 (127.13.121.193:43887)
I20260927 07:00:58.735440 19931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:45753
I20260927 07:00:58.736044 20898 heartbeater.cc:347] Connected to a master server at 127.13.121.252:45047
I20260927 07:00:58.736161 20898 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:58.736523 20898 heartbeater.cc:511] Master 127.13.121.252:45047 requested a full tablet report, sending...
I20260927 07:00:58.738029 20718 ts_manager.cc:194] Registered new tserver with Master: 6011aebe827d4d8cb9596a63325bc278 (127.13.121.193:43887)
I20260927 07:00:58.807624 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 20126
I20260927 07:00:58.821455 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:36965
--local_ip_for_outbound_sockets=127.13.121.194
--tserver_master_addrs=127.13.121.254:44207
--webserver_port=45463
--webserver_interface=127.13.121.194
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab
--principal=oryx/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.121.254:44207,127.13.121.253:39055
--tserver_master_addrs=127.13.121.254:44207,127.13.121.253:39055,127.13.121.252:45047 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:00:59.067054 20771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: NON_VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: true } } }
I20260927 07:00:59.067466 20771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:59.068675 20771 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: attempting to promote NON_VOTER 24cb125cd3904d5e89fd88c34e951906 to VOTER
I20260927 07:00:59.069422 20771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } }
I20260927 07:00:59.071723 20728 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 LEARNER]: Refusing update from remote peer 889e27f4dad54e089c36d203d744b0ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260927 07:00:59.071732 20538 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Refusing update from remote peer 889e27f4dad54e089c36d203d744b0ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260927 07:00:59.072297 20771 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260927 07:00:59.072736 20911 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260927 07:00:59.075836 20912 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:00:59.077644 20728 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:00:59.078958 20912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:00:59.079272 20912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:59.079551 20911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:00:59.079798 20911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:59.080108 20538 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:00:59.081836 20756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:00:59.082093 20756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:00:59.083482 20912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:00:59.083683 20912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab [sys.catalog]: This master's current role is: LEADER
I20260927 07:00:59.087600 20917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "889e27f4dad54e089c36d203d744b0ab" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:00:59.088023 20917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: This master's current role is: FOLLOWER
W20260927 07:00:59.153595 20910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:59.153923 20910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:59.154029 20910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:59.161684 20910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:59.161815 20910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:59.161936 20910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:00:59.172083 20910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab
--principal=oryx/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:36965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=45463
--tserver_master_addrs=127.13.121.254:44207,127.13.121.253:39055,127.13.121.252:45047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:59.173377 20910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:59.174654 20910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:59.188246 20922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:59.188294 20925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:59.188762 20923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:59.189203 20910 server_base.cc:1034] running on GCE node
I20260927 07:00:59.189811 20910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:59.190773 20910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:59.191980 20910 hybrid_clock.cc:648] HybridClock initialized: now 1790492459191916 us; error 51 us; skew 500 ppm
I20260927 07:00:59.192307 20910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:59 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492459, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:59.195384 20910 init.cc:377] Logged in from keytab as oryx/127.13.121.194@KRBTEST.COM (short username oryx)
I20260927 07:00:59.196882 20910 webserver.cc:460] Webserver started at http://127.13.121.194:45463/ using document root <none> and password file <none>
I20260927 07:00:59.197588 20910 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:59.197722 20910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:59.203485 20910 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 07:00:59.206357 20932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:59.207967 20910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:00:59.208135 20910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "ef99443da8a04883ab5bd0928f6ed985"
format_stamp: "Formatted at 2026-09-27 07:00:56 on dist-test-slave-p6hz"
server_key: "3c2c50396a9c563b44112d22487d5c1c"
server_key_iv: "a32d2075440f0ebadc8fb35b3983f582"
server_key_version: "encryptionkey@0"
I20260927 07:00:59.209127 20910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:59.241169 20910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:59.245471 20910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:59.245736 20910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:59.246757 20910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:59.248176 20910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:59.248276 20910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:59.248376 20910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:59.248445 20910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:59.279513 20910 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:36965
I20260927 07:00:59.279574 21044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:36965 every 8 connection(s)
I20260927 07:00:59.280958 20910 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:00:59.281684 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 20910
I20260927 07:00:59.281955 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 20259
Sep 27 07:00:59 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492459, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.252@KRBTEST.COM
Sep 27 07:00:59 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492459, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
Sep 27 07:00:59 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.195@KRBTEST.COM
Sep 27 07:00:59 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492459, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.253@KRBTEST.COM
I20260927 07:00:59.337635 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.285459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:48777 (local address 127.13.121.254:44207)
0927 07:00:59.301718 (+ 16259us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.301727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.306599 (+ 4872us) server_negotiation.cc:407] Connection header received
0927 07:00:59.315261 (+ 8662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.315267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.315356 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.315444 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.317791 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.318938 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.325110 (+ 6172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.325765 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.325892 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.329572 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.329578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.329582 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.329622 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.333653 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.334702 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.334707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.334710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.334875 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.335442 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.335446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.335449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.335748 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.335873 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.337008 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.337242 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16104,"thread_start_us":15995,"threads_started":1}
I20260927 07:00:59.337800 21050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.283849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:45493 (local address 127.13.121.252:45047)
0927 07:00:59.302073 (+ 18224us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.302078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.311253 (+ 9175us) server_negotiation.cc:407] Connection header received
0927 07:00:59.314799 (+ 3546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.314805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.314891 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.314986 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.318191 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.319323 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.321790 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.322477 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.322598 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.329431 (+ 6833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.329438 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.329443 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.330261 (+ 818us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.333509 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.334733 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.334739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.334743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.334887 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.335458 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.335463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.335465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.335795 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.335921 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.337168 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.337391 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18123,"thread_start_us":18007,"threads_started":1}
I20260927 07:00:59.338775 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.307257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.13.121.194:54481)
0927 07:00:59.314864 (+ 7607us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:59.314880 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:59.315004 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:59.315484 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:59.315487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:59.317280 (+ 1793us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:59.317586 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.317611 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.320640 (+ 3029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.320645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.322136 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.322149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.323431 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.323438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.323741 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.324802 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:59.324830 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:59.330675 (+ 5845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:59.334418 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.334422 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.334426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.334950 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.335381 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.335385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.335389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.335541 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.336202 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:59.336206 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:59.336510 (+ 304us) client_negotiation.cc:769] Sending connection context
0927 07:00:59.336815 (+ 305us) client_negotiation.cc:240] Negotiation successful
0927 07:00:59.337372 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7502,"mutex_wait_us":41,"thread_start_us":7384,"threads_started":1}
I20260927 07:00:59.339210 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.284530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.194:48777)
0927 07:00:59.306355 (+ 21825us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:59.306418 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:59.313792 (+ 7374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:59.315615 (+ 1823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:59.315620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:59.316621 (+ 1001us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:59.317579 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.317609 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.322869 (+ 5260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.322874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.324950 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.324964 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.325966 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.325973 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.326165 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.326766 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:59.326792 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:59.329388 (+ 2596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:59.333819 (+ 4431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.333824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.333827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.334567 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.335015 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.335020 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.335035 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.335325 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.335959 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:59.335967 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:59.336592 (+ 625us) client_negotiation.cc:769] Sending connection context
0927 07:00:59.338320 (+ 1728us) client_negotiation.cc:240] Negotiation successful
0927 07:00:59.338672 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21486,"mutex_wait_us":176,"spinlock_wait_cycles":33536,"thread_start_us":21239,"threads_started":1}
I20260927 07:00:59.339224 21049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.283397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46544 (local address 127.13.121.195:36693)
0927 07:00:59.311719 (+ 28322us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.311728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.316344 (+ 4616us) server_negotiation.cc:407] Connection header received
0927 07:00:59.316427 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.316436 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.316707 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.316875 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.319210 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.320828 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.322645 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.323498 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.323629 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.330186 (+ 6557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.330199 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.330216 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.330265 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.333862 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.334715 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.334724 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.334727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.334879 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.335332 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.335336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.335339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.335752 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.335995 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.338521 (+ 2526us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.338806 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28138,"thread_start_us":28131,"threads_started":1}
I20260927 07:00:59.339375 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.285554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:45047 (local address 127.13.121.194:45493)
0927 07:00:59.311118 (+ 25564us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:59.311156 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:59.314180 (+ 3024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:59.315461 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:59.315475 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:59.316714 (+ 1239us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:59.317964 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.317975 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.319512 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.319517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.321609 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.321623 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.324253 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.324259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.324427 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.326127 (+ 1700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:59.326148 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:59.329130 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:59.333755 (+ 4625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.333766 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.333788 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.334589 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.335012 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.335016 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.335020 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.335303 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.335978 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:59.335982 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:59.336507 (+ 525us) client_negotiation.cc:769] Sending connection context
0927 07:00:59.336815 (+ 308us) client_negotiation.cc:240] Negotiation successful
0927 07:00:59.337948 (+ 1133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25431,"mutex_wait_us":1340,"spinlock_wait_cycles":17664,"thread_start_us":25218,"threads_started":1}
I20260927 07:00:59.339680 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.307235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:54481 (local address 127.13.121.253:39055)
0927 07:00:59.314630 (+ 7395us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.314637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.314970 (+ 333us) server_negotiation.cc:407] Connection header received
0927 07:00:59.315119 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.315125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.315220 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.315320 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.319226 (+ 3906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.320490 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.322307 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.322983 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.323103 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.330916 (+ 7813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.330923 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.330928 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.330978 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.334262 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.335078 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.335084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.335087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.335251 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.335657 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.335662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.335665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.336019 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.336141 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.337324 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.337541 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7276,"thread_start_us":7126,"threads_started":1}
I20260927 07:00:59.340680 21046 heartbeater.cc:347] Connected to a master server at 127.13.121.254:44207
I20260927 07:00:59.340739 21047 heartbeater.cc:347] Connected to a master server at 127.13.121.253:39055
I20260927 07:00:59.340809 20382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:00:59.340997 21046 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:59.340996 21047 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:59.341114 21045 heartbeater.cc:347] Connected to a master server at 127.13.121.252:45047
I20260927 07:00:59.341228 21045 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:59.342087 21046 heartbeater.cc:511] Master 127.13.121.254:44207 requested a full tablet report, sending...
I20260927 07:00:59.342113 21047 heartbeater.cc:511] Master 127.13.121.253:39055 requested a full tablet report, sending...
I20260927 07:00:59.342500 21045 heartbeater.cc:511] Master 127.13.121.252:45047 requested a full tablet report, sending...
I20260927 07:00:59.343932 19931 ts_manager.cc:194] Re-registered known tserver with Master: ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965)
I20260927 07:00:59.344070 20718 ts_manager.cc:194] Registered new tserver with Master: ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965)
I20260927 07:00:59.344137 20529 ts_manager.cc:194] Registered new tserver with Master: ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965)
I20260927 07:00:59.345057 19931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:48777
I20260927 07:00:59.427749 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 20259
I20260927 07:00:59.440739 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:36693
--local_ip_for_outbound_sockets=127.13.121.195
--tserver_master_addrs=127.13.121.254:44207
--webserver_port=41743
--webserver_interface=127.13.121.195
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab
--principal=oryx/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.121.254:44207,127.13.121.253:39055
--tserver_master_addrs=127.13.121.254:44207,127.13.121.253:39055,127.13.121.252:45047 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:00:59.738238 20899 heartbeater.cc:503] Master 127.13.121.254:44207 was elected leader, sending a full tablet report...
W20260927 07:00:59.788618 21058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:00:59.788923 21058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:00:59.789063 21058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:00:59.796561 21058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:00:59.796710 21058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:00:59.796994 21058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:00:59.807647 21058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab
--principal=oryx/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:36693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=41743
--tserver_master_addrs=127.13.121.254:44207,127.13.121.253:39055,127.13.121.252:45047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:00:59.809037 21058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:00:59.810375 21058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:00:59.823632 21063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:59.824177 21066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:00:59.824194 21064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:00:59.824705 21058 server_base.cc:1034] running on GCE node
I20260927 07:00:59.825402 21058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:00:59.826429 21058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:00:59.827674 21058 hybrid_clock.cc:648] HybridClock initialized: now 1790492459827618 us; error 45 us; skew 500 ppm
I20260927 07:00:59.828024 21058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:00:59 dist-test-slave-p6hz krb5kdc[19886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492459, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:00:59.831591 21058 init.cc:377] Logged in from keytab as oryx/127.13.121.195@KRBTEST.COM (short username oryx)
I20260927 07:00:59.833308 21058 webserver.cc:460] Webserver started at http://127.13.121.195:41743/ using document root <none> and password file <none>
I20260927 07:00:59.834055 21058 fs_manager.cc:359] Metadata directory not provided
I20260927 07:00:59.834180 21058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:00:59.840443 21058 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260927 07:00:59.843945 21073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:59.845528 21058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:00:59.845695 21058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "6579c7d00e164f199ef48433840423d2"
format_stamp: "Formatted at 2026-09-27 07:00:56 on dist-test-slave-p6hz"
server_key: "a1623b6fbb431c863b9ea10847c36e49"
server_key_iv: "b7e2c2d722e524ee8aa308553d6a9084"
server_key_version: "encryptionkey@0"
I20260927 07:00:59.846717 21058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:00:59.862869 21058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:00:59.867738 21058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:00:59.868011 21058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:00:59.869094 21058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:00:59.870510 21058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:00:59.870599 21058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:59.870698 21058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:00:59.870777 21058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:00:59.901875 21058 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:36693
I20260927 07:00:59.901918 21185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:36693 every 8 connection(s)
I20260927 07:00:59.903668 21058 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:00:59.911876 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 21058
Sep 27 07:00:59 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492459, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.252@KRBTEST.COM
Sep 27 07:00:59 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492459, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.253@KRBTEST.COM
I20260927 07:00:59.960736 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.918715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35084 (local address 127.13.121.254:44207)
0927 07:00:59.924239 (+ 5524us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.924244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.925341 (+ 1097us) server_negotiation.cc:407] Connection header received
0927 07:00:59.925688 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.925693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.925766 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.925871 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.930911 (+ 5040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.932239 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.934174 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.934922 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.935033 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.942905 (+ 7872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.942913 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.942919 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.942971 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.952622 (+ 9651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.955414 (+ 2792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.955420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.955424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.955602 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.957664 (+ 2062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.957671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.957674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.958111 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.958416 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.960061 (+ 1645us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.960325 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5407,"thread_start_us":5303,"threads_started":1}
I20260927 07:00:59.961750 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.907563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:51321 (local address 127.13.121.252:45047)
0927 07:00:59.915550 (+ 7987us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.915554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.926726 (+ 11172us) server_negotiation.cc:407] Connection header received
0927 07:00:59.927734 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.927742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.927831 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.927913 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.941346 (+ 13433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.942606 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.946859 (+ 4253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.947517 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.947653 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.952787 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.952793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.952798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.952842 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.957645 (+ 4803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.958766 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.958772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.958776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.958936 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.959549 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.959554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.959557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.959899 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.960024 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.961136 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.961366 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7869,"thread_start_us":7767,"threads_started":1}
I20260927 07:00:59.964368 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.907593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:45047 (local address 127.13.121.195:51321)
0927 07:00:59.926610 (+ 19017us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:59.926631 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:59.927545 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:59.928620 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:59.928635 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:59.929613 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:59.930638 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.930669 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.945324 (+ 14655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.945332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.946690 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.946710 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.948021 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.948031 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.948413 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.949251 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:59.949287 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:59.952477 (+ 3190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:59.957866 (+ 5389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.957878 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.957903 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.958607 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.959093 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.959097 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.959101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.959390 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.960134 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:59.960143 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:59.960671 (+ 528us) client_negotiation.cc:769] Sending connection context
0927 07:00:59.961968 (+ 1297us) client_negotiation.cc:240] Negotiation successful
0927 07:00:59.962747 (+ 779us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18873,"thread_start_us":10790,"threads_started":1}
I20260927 07:00:59.966152 21186 heartbeater.cc:347] Connected to a master server at 127.13.121.252:45047
I20260927 07:00:59.966514 21186 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:59.967180 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.925752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41548 (local address 127.13.121.253:39055)
0927 07:00:59.941879 (+ 16127us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.941886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.945725 (+ 3839us) server_negotiation.cc:407] Connection header received
0927 07:00:59.945876 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.945881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.945962 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.946063 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.949501 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.950881 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.952761 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.953392 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.953516 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.959363 (+ 5847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.959371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.959376 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.959419 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.962996 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.964163 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.964170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.964173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.964372 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.964840 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.964845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.964848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.965758 (+ 910us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.966014 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.966569 (+ 555us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.966848 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15967,"thread_start_us":16139,"threads_started":1}
I20260927 07:00:59.967499 21186 heartbeater.cc:511] Master 127.13.121.252:45047 requested a full tablet report, sending...
I20260927 07:00:59.969276 20718 ts_manager.cc:194] Registered new tserver with Master: 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693)
I20260927 07:00:59.970000 21200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.919103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38642 (local address 127.13.121.252:45047)
0927 07:00:59.942398 (+ 23295us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.942406 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.942435 (+ 29us) server_negotiation.cc:407] Connection header received
0927 07:00:59.942498 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.942505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.942590 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.942724 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.946031 (+ 3307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.947030 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.949423 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.950072 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.950186 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.955094 (+ 4908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.955111 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.955118 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.955195 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.961251 (+ 6056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.965022 (+ 3771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.965028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.965031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.965213 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.965659 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.965665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.965668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.966008 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.968565 (+ 2557us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.969111 (+ 546us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.969353 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23136,"thread_start_us":26723,"threads_started":1}
Sep 27 07:00:59 dist-test-slave-p6hz krb5kdc[19886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492459, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:00:59.967360 19931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 07:00:59.975217 21193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.907226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.13.121.195:56393)
0927 07:00:59.919501 (+ 12275us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:59.919591 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:59.922790 (+ 3199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:59.933108 (+ 10318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:59.933113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:59.933582 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:59.933831 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.933844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.937616 (+ 3772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.937623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.941127 (+ 3504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.941144 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.950438 (+ 9294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.950447 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.950661 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.953246 (+ 2585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:59.953269 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:59.959178 (+ 5909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:59.972310 (+ 13132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.972316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.972320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.972802 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.973276 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.973281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.973284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.973484 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.974194 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:59.974200 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:59.974345 (+ 145us) client_negotiation.cc:769] Sending connection context
0927 07:00:59.974479 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 07:00:59.974789 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11255,"thread_start_us":19393,"threads_started":1}
I20260927 07:00:59.975328 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.906574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:56393 (local address 127.13.121.253:39055)
0927 07:00:59.907124 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.907156 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.919746 (+ 12590us) server_negotiation.cc:407] Connection header received
0927 07:00:59.932670 (+ 12924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.932676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.932772 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.932853 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.936070 (+ 3217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.937275 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.949219 (+ 11944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.949937 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.950049 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.962862 (+ 12813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.962871 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.962876 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.962921 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.966305 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.972973 (+ 6668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.972980 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.972983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.973153 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.973616 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.973621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.973624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.973994 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.974133 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.974630 (+ 497us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.974896 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":263,"threads_started":1}
I20260927 07:00:59.976261 21188 heartbeater.cc:347] Connected to a master server at 127.13.121.253:39055
I20260927 07:00:59.976411 21188 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:59.976846 21188 heartbeater.cc:511] Master 127.13.121.253:39055 requested a full tablet report, sending...
W20260927 07:00:59.977113 19931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:00:59.978060 20529 ts_manager.cc:194] Registered new tserver with Master: 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693)
I20260927 07:00:59.980144 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.927357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:36407 (local address 127.13.121.254:44207)
0927 07:00:59.945837 (+ 18480us) server_negotiation.cc:206] Beginning negotiation
0927 07:00:59.945844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:00:59.945862 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:00:59.945917 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:00:59.945922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:00:59.945992 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:00:59.946114 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:00:59.951056 (+ 4942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.955573 (+ 4517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.958252 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.958914 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.959054 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.973709 (+ 14655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:00:59.973715 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:00:59.973724 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:00:59.973756 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:00:59.977158 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.977890 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.977896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.977899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.978073 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:00:59.978572 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:00:59.978578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:00:59.978582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:00:59.978881 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:00:59.979005 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 07:00:59.979482 (+ 477us) server_negotiation.cc:299] Negotiation successful
0927 07:00:59.979746 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18343,"thread_start_us":18883,"threads_started":1}
I20260927 07:00:59.980399 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:00:59.927308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.195:36407)
0927 07:00:59.938905 (+ 11597us) negotiation.cc:107] Waiting for socket to connect
0927 07:00:59.938926 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:00:59.939048 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:00:59.946880 (+ 7832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:00:59.946884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:00:59.947493 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:00:59.947846 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.947862 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.955789 (+ 7927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.955796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.957419 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:00:59.957443 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:00:59.959432 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:00:59.959441 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:00:59.959661 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:00:59.962146 (+ 2485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:00:59.962168 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:00:59.973470 (+ 11302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:00:59.977322 (+ 3852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.977326 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.977330 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.977747 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.978267 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:00:59.978272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:00:59.978276 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:00:59.978445 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:00:59.979007 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:00:59.979011 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:00:59.979229 (+ 218us) client_negotiation.cc:769] Sending connection context
0927 07:00:59.979368 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 07:00:59.980035 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11454,"thread_start_us":11849,"threads_started":1}
I20260927 07:00:59.981455 21187 heartbeater.cc:347] Connected to a master server at 127.13.121.254:44207
I20260927 07:00:59.981601 21187 heartbeater.cc:464] Registering TS with master...
I20260927 07:00:59.982080 21187 heartbeater.cc:511] Master 127.13.121.254:44207 requested a full tablet report, sending...
I20260927 07:00:59.983228 19930 ts_manager.cc:194] Re-registered known tserver with Master: 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693)
I20260927 07:00:59.984529 19930 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:36407
I20260927 07:01:00.014463 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.001858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.13.121.195:36693)
0927 07:01:00.003111 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.003118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.004485 (+ 1367us) server_negotiation.cc:407] Connection header received
0927 07:01:00.004592 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.004599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.004861 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.005061 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:00.006686 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.008443 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.011079 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.012798 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.012970 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.013301 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.013990 (+ 689us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.014172 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1103,"thread_start_us":950,"threads_started":1}
I20260927 07:01:00.014456 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.001516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:36693 (local address 127.0.0.1:46546)
0927 07:01:00.004121 (+ 2605us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:00.004141 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:00.004272 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:00.005915 (+ 1643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:00.005924 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:00.005952 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:00.006357 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.006371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.009145 (+ 2774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.009148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.010829 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.010843 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.013601 (+ 2758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.013610 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.013794 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.013821 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 07:01:00.013918 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 07:01:00.014067 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2472,"thread_start_us":2316,"threads_started":1}
I20260927 07:01:00.015892 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.000564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43887 (local address 127.0.0.1:43190)
0927 07:01:00.001588 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:00.001609 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:00.001738 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:00.003871 (+ 2133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:00.003876 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:00.003912 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:00.004301 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.004312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.007007 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.007013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.009012 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.009026 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.015066 (+ 6040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.015073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.015285 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.015320 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:01:00.015384 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 07:01:00.015539 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"thread_start_us":1253,"threads_started":1}
I20260927 07:01:00.016943 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.001104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.13.121.193:43887)
0927 07:01:00.001632 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.001641 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.003070 (+ 1429us) server_negotiation.cc:407] Connection header received
0927 07:01:00.003254 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.003264 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.003591 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.003833 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:00.004474 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.006842 (+ 2368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.011079 (+ 4237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.012745 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.016350 (+ 3605us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.016523 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.016593 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.016724 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":531,"threads_started":1}
I20260927 07:01:00.021981 20833 tablet_service.cc:1467] Processing CreateTablet for tablet 67e03263c0374664b860a1ce384017a9 (DEFAULT_TABLE table=test-table [id=9778372786dd43949ba3ac6d09e9b0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:00.023411 20833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e03263c0374664b860a1ce384017a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:00.023520 21121 tablet_service.cc:1467] Processing CreateTablet for tablet 67e03263c0374664b860a1ce384017a9 (DEFAULT_TABLE table=test-table [id=9778372786dd43949ba3ac6d09e9b0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:00.025262 21121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e03263c0374664b860a1ce384017a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:00.038561 21218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.004442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.13.121.194:36965)
0927 07:01:00.020207 (+ 15765us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.020217 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.020250 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:01:00.020355 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.020363 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.020583 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.020769 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:00.031497 (+ 10728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.033635 (+ 2138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.035920 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.037382 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.037593 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.037858 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.038093 (+ 235us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.038298 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15567,"thread_start_us":14537,"threads_started":1}
I20260927 07:01:00.038609 21219 tablet_bootstrap.cc:492] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: Bootstrap starting.
I20260927 07:01:00.039409 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.004226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:36965 (local address 127.0.0.1:50502)
0927 07:01:00.014725 (+ 10499us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:00.014742 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:00.014833 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:00.020781 (+ 5948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:00.020786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:00.020818 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:00.020993 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.021006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.033839 (+ 12833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.033846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.035703 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.035724 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.037634 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.037643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.037854 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.037891 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 07:01:00.038850 (+ 959us) client_negotiation.cc:240] Negotiation successful
0927 07:01:00.039050 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10396,"thread_start_us":14846,"threads_started":1}
I20260927 07:01:00.039898 21220 tablet_bootstrap.cc:492] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: Bootstrap starting.
I20260927 07:01:00.043653 21220 tablet_bootstrap.cc:654] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:00.043931 20980 tablet_service.cc:1467] Processing CreateTablet for tablet 67e03263c0374664b860a1ce384017a9 (DEFAULT_TABLE table=test-table [id=9778372786dd43949ba3ac6d09e9b0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:00.045063 21220 log.cc:824] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:00.045485 20980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e03263c0374664b860a1ce384017a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:00.049072 21219 tablet_bootstrap.cc:654] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:00.050549 21219 log.cc:824] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:00.056833 21220 tablet_bootstrap.cc:492] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: No bootstrap required, opened a new log
I20260927 07:01:00.057183 21220 ts_tablet_manager.cc:1397] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20260927 07:01:00.061741 21219 tablet_bootstrap.cc:492] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: No bootstrap required, opened a new log
I20260927 07:01:00.062053 21219 ts_tablet_manager.cc:1397] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.006s
I20260927 07:01:00.063895 21223 tablet_bootstrap.cc:492] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: Bootstrap starting.
I20260927 07:01:00.067479 21220 raft_consensus.cc:348] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } }
I20260927 07:01:00.067793 21223 tablet_bootstrap.cc:654] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:00.067883 21220 raft_consensus.cc:374] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:00.067978 21220 raft_consensus.cc:729] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6011aebe827d4d8cb9596a63325bc278, State: Initialized, Role: FOLLOWER
I20260927 07:01:00.068583 21220 consensus_queue.cc:260] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } }
I20260927 07:01:00.069151 21223 log.cc:824] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:00.070358 21220 ts_tablet_manager.cc:1428] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
I20260927 07:01:00.073177 21219 raft_consensus.cc:348] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } }
I20260927 07:01:00.073653 21219 raft_consensus.cc:374] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:00.073762 21219 raft_consensus.cc:729] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6579c7d00e164f199ef48433840423d2, State: Initialized, Role: FOLLOWER
I20260927 07:01:00.074455 21219 consensus_queue.cc:260] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } }
I20260927 07:01:00.075284 21223 tablet_bootstrap.cc:492] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: No bootstrap required, opened a new log
I20260927 07:01:00.075656 21223 ts_tablet_manager.cc:1397] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260927 07:01:00.077798 21219 ts_tablet_manager.cc:1428] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260927 07:01:00.078131 21187 heartbeater.cc:503] Master 127.13.121.254:44207 was elected leader, sending a full tablet report...
I20260927 07:01:00.078826 21224 raft_consensus.cc:482] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:01:00.079613 21224 raft_consensus.cc:504] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } }
I20260927 07:01:00.081245 21224 leader_election.cc:288] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693), ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965)
I20260927 07:01:00.088441 21223 raft_consensus.cc:348] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } }
I20260927 07:01:00.088987 21223 raft_consensus.cc:374] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:00.089093 21223 raft_consensus.cc:729] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef99443da8a04883ab5bd0928f6ed985, State: Initialized, Role: FOLLOWER
I20260927 07:01:00.089816 21223 consensus_queue.cc:260] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } }
I20260927 07:01:00.091571 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.081863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:54549 (local address 127.13.121.195:36693)
0927 07:01:00.082879 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.082887 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.084749 (+ 1862us) server_negotiation.cc:407] Connection header received
0927 07:01:00.085271 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.085277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.085349 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.085433 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:00.086244 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.087370 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.089504 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.090506 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.090667 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.090745 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.091178 (+ 433us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.091318 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831}
I20260927 07:01:00.091917 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.081447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:36693 (local address 127.13.121.193:54549)
0927 07:01:00.084655 (+ 3208us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:00.084676 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:00.085129 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:00.085656 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:00.085662 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:00.085689 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:00.086077 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.086089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.087511 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.087517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.089291 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.089305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.090745 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.090752 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.090948 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.090991 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 07:01:00.091105 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 07:01:00.091311 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3081,"thread_start_us":2890,"threads_started":1}
I20260927 07:01:00.092562 21218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.084927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:51205 (local address 127.13.121.194:36965)
0927 07:01:00.085147 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.085153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.085768 (+ 615us) server_negotiation.cc:407] Connection header received
0927 07:01:00.085975 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.085982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.086050 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.086138 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:00.086603 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.087740 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.090063 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.091214 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.091384 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.091456 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.091967 (+ 511us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.092107 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:01:00.092641 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.084781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:36965 (local address 127.13.121.193:51205)
0927 07:01:00.085667 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:00.085691 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:00.085813 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:00.086165 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:00.086169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:00.086192 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:00.086457 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.086465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.087891 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.087897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.089894 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:00.089908 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.091623 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.091630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:00.091799 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.091833 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 07:01:00.091932 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 07:01:00.092114 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":610,"threads_started":1}
I20260927 07:01:00.093482 21046 heartbeater.cc:503] Master 127.13.121.254:44207 was elected leader, sending a full tablet report...
I20260927 07:01:00.093400 21141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e03263c0374664b860a1ce384017a9" candidate_uuid: "6011aebe827d4d8cb9596a63325bc278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6579c7d00e164f199ef48433840423d2" is_pre_election: true
I20260927 07:01:00.093953 21141 raft_consensus.cc:2454] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6011aebe827d4d8cb9596a63325bc278 in term 0.
I20260927 07:01:00.094362 21223 ts_tablet_manager.cc:1428] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: Time spent starting tablet: real 0.019s user 0.011s sys 0.004s
I20260927 07:01:00.095402 20785 leader_election.cc:302] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6011aebe827d4d8cb9596a63325bc278, 6579c7d00e164f199ef48433840423d2; no voters:
I20260927 07:01:00.095888 21224 raft_consensus.cc:2781] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:01:00.096057 21224 raft_consensus.cc:482] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:01:00.096148 21224 raft_consensus.cc:3037] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:00.096740 21000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e03263c0374664b860a1ce384017a9" candidate_uuid: "6011aebe827d4d8cb9596a63325bc278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef99443da8a04883ab5bd0928f6ed985" is_pre_election: true
I20260927 07:01:00.097301 21000 raft_consensus.cc:2454] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6011aebe827d4d8cb9596a63325bc278 in term 0.
I20260927 07:01:00.101413 21224 raft_consensus.cc:504] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } }
I20260927 07:01:00.102134 21224 leader_election.cc:288] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [CANDIDATE]: Term 1 election: Requested vote from peers 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693), ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965)
I20260927 07:01:00.103026 21000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e03263c0374664b860a1ce384017a9" candidate_uuid: "6011aebe827d4d8cb9596a63325bc278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef99443da8a04883ab5bd0928f6ed985"
I20260927 07:01:00.103299 21000 raft_consensus.cc:3037] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:00.103585 21141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e03263c0374664b860a1ce384017a9" candidate_uuid: "6011aebe827d4d8cb9596a63325bc278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6579c7d00e164f199ef48433840423d2"
I20260927 07:01:00.103801 21141 raft_consensus.cc:3037] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:00.106114 21000 raft_consensus.cc:2454] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6011aebe827d4d8cb9596a63325bc278 in term 1.
I20260927 07:01:00.106236 21141 raft_consensus.cc:2454] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6011aebe827d4d8cb9596a63325bc278 in term 1.
I20260927 07:01:00.106570 20785 leader_election.cc:302] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6011aebe827d4d8cb9596a63325bc278, ef99443da8a04883ab5bd0928f6ed985; no voters:
I20260927 07:01:00.106904 21224 raft_consensus.cc:2781] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:00.107393 21224 raft_consensus.cc:686] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 1 LEADER]: Becoming Leader. State: Replica: 6011aebe827d4d8cb9596a63325bc278, State: Running, Role: LEADER
I20260927 07:01:00.107939 21224 consensus_queue.cc:237] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } }
I20260927 07:01:00.112834 19931 catalog_manager.cc:5495] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6011aebe827d4d8cb9596a63325bc278 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "6011aebe827d4d8cb9596a63325bc278" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } health_report { overall_health: UNKNOWN } } }
I20260927 07:01:00.139639 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.134278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.13.121.193:43887)
0927 07:01:00.134467 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.134472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.134491 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:01:00.134598 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.134602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.134684 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.134806 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:00.135232 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.135973 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.137347 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.137839 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.137955 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.138439 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:00.138962 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:00.139072 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.139245 (+ 173us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.139335 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 07:01:00.147948 21000 raft_consensus.cc:1261] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [term 1 FOLLOWER]: Refusing update from remote peer 6011aebe827d4d8cb9596a63325bc278: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:00.147950 21141 raft_consensus.cc:1261] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [term 1 FOLLOWER]: Refusing update from remote peer 6011aebe827d4d8cb9596a63325bc278: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:00.148927 21224 consensus_queue.cc:1035] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:00.149269 21230 consensus_queue.cc:1035] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:00.183964 19929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:00.184249 19929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:00.191309 19929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 889e27f4dad54e089c36d203d744b0ab: Sending DeleteTablet for 3 replicas of tablet 67e03263c0374664b860a1ce384017a9
I20260927 07:01:00.192956 21121 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e03263c0374664b860a1ce384017a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:00 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:46546
I20260927 07:01:00.193006 20980 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e03263c0374664b860a1ce384017a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:00 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:50502
I20260927 07:01:00.192989 20833 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e03263c0374664b860a1ce384017a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:00 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:43190
I20260927 07:01:00.193879 21244 tablet_replica.cc:330] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: stopping tablet replica
I20260927 07:01:00.194424 21244 raft_consensus.cc:2229] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:00.195077 21244 raft_consensus.cc:2258] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:00.196491 21246 tablet_replica.cc:330] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: stopping tablet replica
I20260927 07:01:00.196919 21246 raft_consensus.cc:2229] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:01:00.197621 21244 ts_tablet_manager.cc:1905] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:00.197629 21246 raft_consensus.cc:2258] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:00.198563 21245 tablet_replica.cc:330] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: stopping tablet replica
I20260927 07:01:00.200977 21246 ts_tablet_manager.cc:1905] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:00.203738 21245 raft_consensus.cc:2229] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:00.204298 21245 raft_consensus.cc:2258] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:00.204998 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 19899
I20260927 07:01:00.205751 21244 ts_tablet_manager.cc:1918] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:00.206053 21244 log.cc:1196] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/67e03263c0374664b860a1ce384017a9
I20260927 07:01:00.206110 19961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:00.206671 21244 ts_tablet_manager.cc:1939] T 67e03263c0374664b860a1ce384017a9 P 6579c7d00e164f199ef48433840423d2: Deleting consensus metadata
I20260927 07:01:00.208647 21245 ts_tablet_manager.cc:1905] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:00.208760 19916 catalog_manager.cc:4841] TS 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693): tablet 67e03263c0374664b860a1ce384017a9 (table test-table [id=9778372786dd43949ba3ac6d09e9b0fd]) successfully deleted
I20260927 07:01:00.214592 21245 ts_tablet_manager.cc:1918] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:00.214782 21245 log.cc:1196] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/67e03263c0374664b860a1ce384017a9
I20260927 07:01:00.215379 21245 ts_tablet_manager.cc:1939] T 67e03263c0374664b860a1ce384017a9 P ef99443da8a04883ab5bd0928f6ed985: Deleting consensus metadata
I20260927 07:01:00.215582 21246 ts_tablet_manager.cc:1918] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:00.215806 21246 log.cc:1196] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/67e03263c0374664b860a1ce384017a9
I20260927 07:01:00.216421 21246 ts_tablet_manager.cc:1939] T 67e03263c0374664b860a1ce384017a9 P 6011aebe827d4d8cb9596a63325bc278: Deleting consensus metadata
I20260927 07:01:00.232534 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.195165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41550 (local address 127.13.121.253:39055)
0927 07:01:00.195345 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.195352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.204058 (+ 8706us) server_negotiation.cc:407] Connection header received
0927 07:01:00.211332 (+ 7274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.211340 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.211469 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.211556 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:00.214505 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.215768 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.218097 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.218709 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.218845 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.225750 (+ 6905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:00.225759 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:00.225764 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:00.225820 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:00.229251 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:00.230200 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:00.230206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:00.230209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:00.230360 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:00.230769 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:00.230773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:00.230776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:00.231128 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:00.231387 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.231890 (+ 503us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.232166 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:01:00.250730 21200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.204216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.13.121.252:45047)
0927 07:01:00.204395 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.204400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.233505 (+ 29105us) server_negotiation.cc:407] Connection header received
0927 07:01:00.233767 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.233774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.233857 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.233929 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:00.235837 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.236915 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.238877 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.239588 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.239714 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.243343 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:00.243351 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:00.243356 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:00.243408 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:00.247006 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:00.247976 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:00.247982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:00.247987 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:00.248153 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:00.248572 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:00.248577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:00.248580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:00.248957 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:00.249230 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.250138 (+ 908us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.250387 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:01:00.361164 19917 catalog_manager.cc:4841] TS 6011aebe827d4d8cb9596a63325bc278 (127.13.121.193:43887): tablet 67e03263c0374664b860a1ce384017a9 (table test-table [id=9778372786dd43949ba3ac6d09e9b0fd]) successfully deleted
I20260927 07:01:00.361202 19916 catalog_manager.cc:4841] TS ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965): tablet 67e03263c0374664b860a1ce384017a9 (table test-table [id=9778372786dd43949ba3ac6d09e9b0fd]) successfully deleted
W20260927 07:01:00.361472 19917 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.13.121.193:43887, user_credentials={real_user=slave}} blocked reactor thread for 143388us
W20260927 07:01:00.361678 19916 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.13.121.194:36965, user_credentials={real_user=slave}} blocked reactor thread for 144334us
I20260927 07:01:00.361923 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 19899
I20260927 07:01:00.402755 21200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.386901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.13.121.252:45047)
0927 07:01:00.387116 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.387122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.388291 (+ 1169us) server_negotiation.cc:407] Connection header received
0927 07:01:00.388631 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.388638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.388729 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.388854 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:00.390810 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.391630 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.393075 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.393495 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.393611 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.396711 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:00.396717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:00.396722 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:00.396765 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:00.399447 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:00.400275 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:00.400281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:00.400286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:00.400445 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:00.401063 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:00.401070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:00.401073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:00.401464 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:00.401703 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.402089 (+ 386us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.402310 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:01:00.405727 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:00.387980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41558 (local address 127.13.121.253:39055)
0927 07:01:00.388151 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:00.388157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:00.389624 (+ 1467us) server_negotiation.cc:407] Connection header received
0927 07:01:00.389782 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:00.389788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:00.389883 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:00.389992 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:00.392140 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.393272 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.395341 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:00.396013 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:00.396163 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:00.399290 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:00.399299 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:00.399305 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:00.399356 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:00.402346 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:00.403489 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:00.403495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:00.403498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:00.403651 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:00.404255 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:00.404259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:00.404261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:00.404580 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:00.404761 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:00.405234 (+ 473us) server_negotiation.cc:299] Negotiation successful
0927 07:01:00.405416 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:01:01.174042 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.173273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.195:46155)
0927 07:01:01.173706 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.173824 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":228,"threads_started":1}
W20260927 07:01:01.174590 21187 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:44207 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
I20260927 07:01:01.179446 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.178379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:49321)
0927 07:01:01.179064 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.179237 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":770,"threads_started":1}
I20260927 07:01:01.179769 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.179030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.194:52343)
0927 07:01:01.179464 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.179592 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":436,"threads_started":1}
W20260927 07:01:01.180258 20899 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:44207 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
W20260927 07:01:01.180248 21046 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:44207 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
I20260927 07:01:01.691701 21258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 889e27f4dad54e089c36d203d744b0ab)
I20260927 07:01:01.691962 21258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } }
I20260927 07:01:01.692405 21259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 889e27f4dad54e089c36d203d744b0ab)
I20260927 07:01:01.692569 21259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } }
I20260927 07:01:01.693320 21259 leader_election.cc:288] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207), 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252:45047)
I20260927 07:01:01.693364 21258 leader_election.cc:288] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207), 4c72c27da588488b9d00ed0d698884bc (127.13.121.253:39055)
I20260927 07:01:01.694437 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.693495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35604)
0927 07:01:01.694034 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.694182 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":232,"threads_started":1}
I20260927 07:01:01.694790 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.693512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35612)
0927 07:01:01.694332 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.694519 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":547,"threads_started":1}
W20260927 07:01:01.694772 20515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
I20260927 07:01:01.696795 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.696362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35620)
0927 07:01:01.696581 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.696657 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260927 07:01:01.696815 20704 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
W20260927 07:01:01.697062 20515 leader_election.cc:334] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207): Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
I20260927 07:01:01.705266 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.694388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:39055 (local address 127.0.0.1:41560)
0927 07:01:01.695160 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.695180 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:01.695323 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:01.697552 (+ 2229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:01.697559 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:01.697591 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:01.697849 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:01.697861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:01.699901 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:01.699908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:01.702345 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:01.702369 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:01.704374 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:01.704382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:01.704613 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:01.704658 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 07:01:01.704731 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 07:01:01.704902 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":2129,"threads_started":1}
I20260927 07:01:01.705338 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.696363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:45047 (local address 127.0.0.1:38656)
0927 07:01:01.697515 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.697536 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:01.697690 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:01.698311 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:01.698317 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:01.698342 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:01.698659 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:01.698669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:01.700364 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:01.700371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:01.702553 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:01.702573 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:01.704470 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:01.704478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:01.704699 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:01.704743 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 07:01:01.704812 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 07:01:01.704992 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1057,"mutex_wait_us":552,"threads_started":1}
I20260927 07:01:01.706276 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.693949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38656 (local address 127.13.121.252:45047)
0927 07:01:01.697028 (+ 3079us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:01.697033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:01.697799 (+ 766us) server_negotiation.cc:407] Connection header received
0927 07:01:01.697880 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:01.697886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:01.697958 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:01.698042 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:01.698843 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:01.700170 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:01.702806 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:01.704228 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:01.705625 (+ 1397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:01.705706 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:01.705762 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 07:01:01.705900 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2971,"thread_start_us":2863,"threads_started":1}
I20260927 07:01:01.707203 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.694670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41560 (local address 127.13.121.253:39055)
0927 07:01:01.697161 (+ 2491us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:01.697167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:01.697191 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:01:01.697274 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:01.697280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:01.697365 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:01.697988 (+ 623us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:01.698221 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:01.699678 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:01.702674 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:01.704122 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:01.706171 (+ 2049us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:01.706254 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:01.706306 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 07:01:01.706846 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1402,"thread_start_us":1240,"threads_started":1}
I20260927 07:01:01.707602 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.707174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35636)
0927 07:01:01.707347 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.707443 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 07:01:01.707983 20728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c72c27da588488b9d00ed0d698884bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "24cb125cd3904d5e89fd88c34e951906" is_pre_election: true
I20260927 07:01:01.707983 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "24cb125cd3904d5e89fd88c34e951906" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4c72c27da588488b9d00ed0d698884bc" is_pre_election: true
I20260927 07:01:01.708289 20728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c72c27da588488b9d00ed0d698884bc in term 1.
I20260927 07:01:01.708283 20538 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24cb125cd3904d5e89fd88c34e951906 in term 1.
W20260927 07:01:01.708554 20704 leader_election.cc:334] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207): Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
I20260927 07:01:01.708770 20514 leader_election.cc:302] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24cb125cd3904d5e89fd88c34e951906, 4c72c27da588488b9d00ed0d698884bc; no voters: 889e27f4dad54e089c36d203d744b0ab
I20260927 07:01:01.709000 20704 leader_election.cc:302] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24cb125cd3904d5e89fd88c34e951906, 4c72c27da588488b9d00ed0d698884bc; no voters: 889e27f4dad54e089c36d203d744b0ab
I20260927 07:01:01.709086 21259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 07:01:01.709203 21259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 889e27f4dad54e089c36d203d744b0ab)
I20260927 07:01:01.709288 21259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:01:01.709326 21258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 07:01:01.709434 21258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 889e27f4dad54e089c36d203d744b0ab)
I20260927 07:01:01.709527 21258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:01:01.712443 21258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } }
I20260927 07:01:01.712443 21259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } }
I20260927 07:01:01.713158 21258 leader_election.cc:288] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [CANDIDATE]: Term 2 election: Requested vote from peers 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207), 4c72c27da588488b9d00ed0d698884bc (127.13.121.253:39055)
I20260927 07:01:01.713195 21259 leader_election.cc:288] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 2 election: Requested vote from peers 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207), 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252:45047)
I20260927 07:01:01.713785 20728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c72c27da588488b9d00ed0d698884bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "24cb125cd3904d5e89fd88c34e951906"
I20260927 07:01:01.714071 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.713634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35648)
0927 07:01:01.713805 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.713922 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260927 07:01:01.714143 20728 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4c72c27da588488b9d00ed0d698884bc in current term 2: Already voted for candidate 24cb125cd3904d5e89fd88c34e951906 in this term.
I20260927 07:01:01.714212 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.713370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35642)
0927 07:01:01.713761 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.713979 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254}
I20260927 07:01:01.714177 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "24cb125cd3904d5e89fd88c34e951906" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4c72c27da588488b9d00ed0d698884bc"
I20260927 07:01:01.714475 20538 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 24cb125cd3904d5e89fd88c34e951906 in current term 2: Already voted for candidate 4c72c27da588488b9d00ed0d698884bc in this term.
I20260927 07:01:01.715229 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.714869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35660)
0927 07:01:01.715025 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.715090 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260927 07:01:01.715394 20515 leader_election.cc:334] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207): Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
I20260927 07:01:01.715510 20515 leader_election.cc:302] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c72c27da588488b9d00ed0d698884bc; no voters: 24cb125cd3904d5e89fd88c34e951906, 889e27f4dad54e089c36d203d744b0ab
I20260927 07:01:01.715554 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:01.715182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35668)
0927 07:01:01.715343 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:01.715423 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260927 07:01:01.715755 21259 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260927 07:01:01.715790 20704 leader_election.cc:334] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207): Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
I20260927 07:01:01.715955 20704 leader_election.cc:302] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24cb125cd3904d5e89fd88c34e951906; no voters: 4c72c27da588488b9d00ed0d698884bc, 889e27f4dad54e089c36d203d744b0ab
I20260927 07:01:01.716239 21258 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 07:01:02.176745 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:02.175546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.195:42441)
0927 07:01:02.176396 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:02.176508 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":421,"threads_started":1}
I20260927 07:01:02.182042 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:02.181021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.194:58043)
0927 07:01:02.181640 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:02.181768 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":325,"threads_started":1}
I20260927 07:01:02.183310 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:02.181463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:47319)
0927 07:01:02.182839 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:02.183006 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1234,"thread_start_us":1121,"threads_started":1}
I20260927 07:01:03.178656 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.177848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.195:49989)
0927 07:01:03.178354 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.178456 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":259,"threads_started":1}
I20260927 07:01:03.184916 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.183502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.194:36647)
0927 07:01:03.184023 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.184137 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":244,"threads_started":1}
I20260927 07:01:03.185637 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.184791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:36463)
0927 07:01:03.185375 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.185463 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":361,"threads_started":1}
I20260927 07:01:03.454808 21274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:01:03.455056 21274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } }
I20260927 07:01:03.455765 21274 leader_election.cc:288] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207), 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252:45047)
I20260927 07:01:03.456475 20728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c72c27da588488b9d00ed0d698884bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "24cb125cd3904d5e89fd88c34e951906" is_pre_election: true
I20260927 07:01:03.456725 20728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c72c27da588488b9d00ed0d698884bc in term 2.
I20260927 07:01:03.456820 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.455935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35930)
0927 07:01:03.456500 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.456621 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":257,"threads_started":1}
I20260927 07:01:03.457383 20514 leader_election.cc:302] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24cb125cd3904d5e89fd88c34e951906, 4c72c27da588488b9d00ed0d698884bc; no voters:
I20260927 07:01:03.457722 21274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260927 07:01:03.457864 21274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:01:03.457962 21274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 2 FOLLOWER]: Advancing to term 3
I20260927 07:01:03.458734 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.458336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35938)
0927 07:01:03.458516 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.458592 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260927 07:01:03.458935 20515 leader_election.cc:334] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207): Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
I20260927 07:01:03.460395 21274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } }
I20260927 07:01:03.460970 21274 leader_election.cc:288] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 3 election: Requested vote from peers 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207), 24cb125cd3904d5e89fd88c34e951906 (127.13.121.252:45047)
I20260927 07:01:03.461584 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.461145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35940)
0927 07:01:03.461302 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.461397 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 07:01:03.461603 20728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c72c27da588488b9d00ed0d698884bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "24cb125cd3904d5e89fd88c34e951906"
I20260927 07:01:03.461813 20728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 2 FOLLOWER]: Advancing to term 3
I20260927 07:01:03.462634 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.462242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35946)
0927 07:01:03.462408 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.462502 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260927 07:01:03.462836 20515 leader_election.cc:334] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207): Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
I20260927 07:01:03.464344 20728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c72c27da588488b9d00ed0d698884bc in term 3.
I20260927 07:01:03.464843 20514 leader_election.cc:302] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24cb125cd3904d5e89fd88c34e951906, 4c72c27da588488b9d00ed0d698884bc; no voters: 889e27f4dad54e089c36d203d744b0ab
I20260927 07:01:03.465240 21274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 3 FOLLOWER]: Leader election won for term 3
I20260927 07:01:03.465638 21274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [term 3 LEADER]: Becoming Leader. State: Replica: 4c72c27da588488b9d00ed0d698884bc, State: Running, Role: LEADER
I20260927 07:01:03.466060 21274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } }
I20260927 07:01:03.468832 21277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c72c27da588488b9d00ed0d698884bc. Latest consensus state: current_term: 3 leader_uuid: "4c72c27da588488b9d00ed0d698884bc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:01:03.469099 21277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:03.469784 21279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:01:03.473621 21279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:01:03.474519 21279 catalog_manager.cc:1259] Loaded cluster ID: da751584e4034fd3b53ed39c1ae55e43
I20260927 07:01:03.474607 21279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:01:03.475831 21279 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:01:03.476673 21279 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Loaded TSK: 0
I20260927 07:01:03.477533 21279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:01:03.531092 20528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:01:03.538403 20528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:01:03.542892 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.542376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35972)
0927 07:01:03.542586 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.542685 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260927 07:01:03.542977 20728 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [term 3 FOLLOWER]: Refusing update from remote peer 4c72c27da588488b9d00ed0d698884bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260927 07:01:03.543705 21277 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260927 07:01:03.543910 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.543593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35974)
0927 07:01:03.543739 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.543807 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 07:01:03.544068 20515 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc -> Peer 889e27f4dad54e089c36d203d744b0ab (127.13.121.254:44207): Couldn't send request to peer 889e27f4dad54e089c36d203d744b0ab. Status: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 07:01:03.546087 21281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c72c27da588488b9d00ed0d698884bc. Latest consensus state: current_term: 3 leader_uuid: "4c72c27da588488b9d00ed0d698884bc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:01:03.546343 21281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:01:03.547856 21277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "4c72c27da588488b9d00ed0d698884bc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:01:03.548139 21277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:03.548130 21282 mvcc.cc:205] Tried to move back new op lower bound from 7333857130664296448 to 7333857130363441152. Current Snapshot: MvccSnapshot[applied={T|T < 7333857130664296448}]
I20260927 07:01:03.548770 21281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "4c72c27da588488b9d00ed0d698884bc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "889e27f4dad54e089c36d203d744b0ab" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44207 } } peers { permanent_uuid: "4c72c27da588488b9d00ed0d698884bc" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 39055 } attrs { promote: false } } peers { permanent_uuid: "24cb125cd3904d5e89fd88c34e951906" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 45047 } attrs { promote: false } } }
I20260927 07:01:03.549027 21281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb125cd3904d5e89fd88c34e951906 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:01:03.564329 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.556237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:36965 (local address 127.0.0.1:50508)
0927 07:01:03.556831 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.556853 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:03.556994 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:03.557566 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:03.557573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:03.557600 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:03.557883 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:03.557895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.559547 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.559554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:03.561657 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:03.561679 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.563497 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.563504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:03.563699 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:03.563742 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 07:01:03.563811 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 07:01:03.563973 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460}
I20260927 07:01:03.565851 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.556482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50508 (local address 127.13.121.194:36965)
0927 07:01:03.557243 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:03.557249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:03.557268 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:01:03.557335 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:03.557340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:03.557415 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:03.558029 (+ 614us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:03.558233 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.559362 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.561911 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.563250 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.565213 (+ 1963us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:03.565297 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:03.565354 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 07:01:03.565503 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":646,"threads_started":1}
I20260927 07:01:03.571108 20980 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9d27a9dc33440b9f57b44176610c9f (DEFAULT_TABLE table=test-table [id=d9ec2ecb4b9e40b180d3a22faa581986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:03.571616 20980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9d27a9dc33440b9f57b44176610c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:03.574587 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.557108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43887 (local address 127.0.0.1:43202)
0927 07:01:03.566531 (+ 9423us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.566552 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:03.566673 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:03.567722 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:03.567728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:03.567758 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:03.568143 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:03.568154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.569787 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.569793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:03.572214 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:03.572227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.573731 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.573737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:03.573916 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:03.573951 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:01:03.574029 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 07:01:03.574172 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9301,"mutex_wait_us":1,"thread_start_us":9167,"threads_started":1}
I20260927 07:01:03.574676 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.557498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.13.121.193:43887)
0927 07:01:03.567388 (+ 9890us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:03.567396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:03.567416 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:01:03.567487 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:03.567492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:03.567565 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:03.568319 (+ 754us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:03.568505 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.569603 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.572369 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.573546 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.573712 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:03.574191 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:03.574242 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 07:01:03.574378 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9766,"thread_start_us":9642,"threads_started":1}
I20260927 07:01:03.575516 20833 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9d27a9dc33440b9f57b44176610c9f (DEFAULT_TABLE table=test-table [id=d9ec2ecb4b9e40b180d3a22faa581986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:03.575968 20833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9d27a9dc33440b9f57b44176610c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:03.576934 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.556522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:36693 (local address 127.0.0.1:46548)
0927 07:01:03.564864 (+ 8342us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.564886 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:03.565017 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:03.571547 (+ 6530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:03.571552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:03.571577 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:03.571697 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:03.571704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.572881 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.572886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:03.574496 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:03.574508 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.576201 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.576207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:03.576385 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:03.576421 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 07:01:03.576517 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 07:01:03.576656 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8225,"thread_start_us":13548,"threads_started":1}
I20260927 07:01:03.576999 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.556682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.13.121.195:36693)
0927 07:01:03.571242 (+ 14560us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:03.571248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:03.571268 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:01:03.571331 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:03.571336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:03.571415 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:03.571515 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:03.571831 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.572760 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.574907 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.576017 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.576176 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:03.576244 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:03.576565 (+ 321us) server_negotiation.cc:299] Negotiation successful
0927 07:01:03.576705 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14425,"thread_start_us":14320,"threads_started":1}
I20260927 07:01:03.578088 21292 tablet_bootstrap.cc:492] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: Bootstrap starting.
I20260927 07:01:03.579553 21121 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9d27a9dc33440b9f57b44176610c9f (DEFAULT_TABLE table=test-table [id=d9ec2ecb4b9e40b180d3a22faa581986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:03.579754 21292 tablet_bootstrap.cc:654] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:03.579977 21121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9d27a9dc33440b9f57b44176610c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:03.582031 21292 tablet_bootstrap.cc:492] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: No bootstrap required, opened a new log
I20260927 07:01:03.582194 21292 ts_tablet_manager.cc:1397] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260927 07:01:03.583007 21292 raft_consensus.cc:348] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } }
I20260927 07:01:03.583304 21292 raft_consensus.cc:374] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:03.583701 21292 raft_consensus.cc:729] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef99443da8a04883ab5bd0928f6ed985, State: Initialized, Role: FOLLOWER
I20260927 07:01:03.584288 21292 consensus_queue.cc:260] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } }
I20260927 07:01:03.588393 21294 tablet_bootstrap.cc:492] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: Bootstrap starting.
I20260927 07:01:03.590500 21294 tablet_bootstrap.cc:654] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:03.591753 21292 ts_tablet_manager.cc:1428] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
I20260927 07:01:03.591993 21296 tablet_bootstrap.cc:492] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: Bootstrap starting.
I20260927 07:01:03.593024 21294 tablet_bootstrap.cc:492] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: No bootstrap required, opened a new log
I20260927 07:01:03.593765 21294 ts_tablet_manager.cc:1397] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260927 07:01:03.593793 21296 tablet_bootstrap.cc:654] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:03.594414 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.593312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.194:54813)
0927 07:01:03.594158 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.594257 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":731}
I20260927 07:01:03.595733 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.595274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.194:41497)
0927 07:01:03.595493 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.595580 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260927 07:01:03.596845 21047 heartbeater.cc:503] Master 127.13.121.253:39055 was elected leader, sending a full tablet report...
I20260927 07:01:03.597394 21294 raft_consensus.cc:348] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } }
I20260927 07:01:03.597643 21294 raft_consensus.cc:374] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:03.597740 21294 raft_consensus.cc:729] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6011aebe827d4d8cb9596a63325bc278, State: Initialized, Role: FOLLOWER
I20260927 07:01:03.597956 21294 consensus_queue.cc:260] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } }
I20260927 07:01:03.598760 21294 ts_tablet_manager.cc:1428] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260927 07:01:03.599591 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.599052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:33209)
0927 07:01:03.599404 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.599476 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246}
I20260927 07:01:03.601132 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.600742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:52561)
0927 07:01:03.600906 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.600984 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260927 07:01:03.601861 20900 heartbeater.cc:503] Master 127.13.121.253:39055 was elected leader, sending a full tablet report...
I20260927 07:01:03.601992 21296 tablet_bootstrap.cc:492] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: No bootstrap required, opened a new log
I20260927 07:01:03.602110 21296 ts_tablet_manager.cc:1397] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260927 07:01:03.602936 21296 raft_consensus.cc:348] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } }
I20260927 07:01:03.603235 21296 raft_consensus.cc:374] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:03.603336 21296 raft_consensus.cc:729] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6579c7d00e164f199ef48433840423d2, State: Initialized, Role: FOLLOWER
I20260927 07:01:03.603621 21296 consensus_queue.cc:260] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } }
I20260927 07:01:03.604499 21296 ts_tablet_manager.cc:1428] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 07:01:03.605115 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.604716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.195:56155)
0927 07:01:03.604860 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.604950 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 07:01:03.606205 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.605790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.195:36791)
0927 07:01:03.605944 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.606038 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 07:01:03.606853 21188 heartbeater.cc:503] Master 127.13.121.253:39055 was elected leader, sending a full tablet report...
W20260927 07:01:03.666527 21189 tablet.cc:2369] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:01:03.692656 21299 raft_consensus.cc:482] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:01:03.692903 21299 raft_consensus.cc:504] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } }
I20260927 07:01:03.693697 21299 leader_election.cc:288] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965), 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693)
I20260927 07:01:03.694243 21000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9d27a9dc33440b9f57b44176610c9f" candidate_uuid: "6011aebe827d4d8cb9596a63325bc278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef99443da8a04883ab5bd0928f6ed985" is_pre_election: true
I20260927 07:01:03.694280 21141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9d27a9dc33440b9f57b44176610c9f" candidate_uuid: "6011aebe827d4d8cb9596a63325bc278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6579c7d00e164f199ef48433840423d2" is_pre_election: true
I20260927 07:01:03.694540 21000 raft_consensus.cc:2454] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6011aebe827d4d8cb9596a63325bc278 in term 0.
I20260927 07:01:03.694542 21141 raft_consensus.cc:2454] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6011aebe827d4d8cb9596a63325bc278 in term 0.
I20260927 07:01:03.695022 20785 leader_election.cc:302] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6011aebe827d4d8cb9596a63325bc278, ef99443da8a04883ab5bd0928f6ed985; no voters:
I20260927 07:01:03.695307 21299 raft_consensus.cc:2781] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:01:03.695432 21299 raft_consensus.cc:482] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:01:03.695521 21299 raft_consensus.cc:3037] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:03.697538 21299 raft_consensus.cc:504] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } }
I20260927 07:01:03.698081 21299 leader_election.cc:288] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [CANDIDATE]: Term 1 election: Requested vote from peers ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965), 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693)
I20260927 07:01:03.698652 21000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9d27a9dc33440b9f57b44176610c9f" candidate_uuid: "6011aebe827d4d8cb9596a63325bc278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef99443da8a04883ab5bd0928f6ed985"
I20260927 07:01:03.698690 21141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9d27a9dc33440b9f57b44176610c9f" candidate_uuid: "6011aebe827d4d8cb9596a63325bc278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6579c7d00e164f199ef48433840423d2"
I20260927 07:01:03.698833 21000 raft_consensus.cc:3037] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:03.698904 21141 raft_consensus.cc:3037] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:03.700973 21000 raft_consensus.cc:2454] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6011aebe827d4d8cb9596a63325bc278 in term 1.
I20260927 07:01:03.701280 21141 raft_consensus.cc:2454] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6011aebe827d4d8cb9596a63325bc278 in term 1.
I20260927 07:01:03.701493 20785 leader_election.cc:302] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6011aebe827d4d8cb9596a63325bc278, ef99443da8a04883ab5bd0928f6ed985; no voters:
I20260927 07:01:03.701828 21299 raft_consensus.cc:2781] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:03.702207 21299 raft_consensus.cc:686] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 1 LEADER]: Becoming Leader. State: Replica: 6011aebe827d4d8cb9596a63325bc278, State: Running, Role: LEADER
I20260927 07:01:03.702488 21299 consensus_queue.cc:237] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } }
I20260927 07:01:03.703300 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.702884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:37259)
0927 07:01:03.703048 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.703128 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 07:01:03.705209 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.704753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:41447)
0927 07:01:03.704912 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.705047 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 07:01:03.705874 20528 catalog_manager.cc:5495] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6011aebe827d4d8cb9596a63325bc278 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "6011aebe827d4d8cb9596a63325bc278" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6011aebe827d4d8cb9596a63325bc278" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43887 } health_report { overall_health: HEALTHY } } }
W20260927 07:01:03.710762 20901 tablet.cc:2369] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:01:03.739161 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.734475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.13.121.193:43887)
0927 07:01:03.734625 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:03.734630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:03.734647 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:01:03.734786 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:03.734789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:03.734850 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:03.734925 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:03.735248 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.736026 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.737417 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.737890 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.737990 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:03.738448 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:03.738574 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:03.738640 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:03.738839 (+ 199us) server_negotiation.cc:299] Negotiation successful
0927 07:01:03.738915 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:01:03.742170 21141 raft_consensus.cc:1261] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [term 1 FOLLOWER]: Refusing update from remote peer 6011aebe827d4d8cb9596a63325bc278: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:03.742231 21000 raft_consensus.cc:1261] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [term 1 FOLLOWER]: Refusing update from remote peer 6011aebe827d4d8cb9596a63325bc278: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:03.742780 21301 consensus_queue.cc:1035] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6579c7d00e164f199ef48433840423d2" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:03.743095 21299 consensus_queue.cc:1035] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef99443da8a04883ab5bd0928f6ed985" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 36965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:03.745561 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.744958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.195:46599)
0927 07:01:03.745281 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.745396 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260927 07:01:03.746091 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.745015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.194:44673)
0927 07:01:03.745820 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.745921 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":684}
I20260927 07:01:03.751960 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.751324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.194:36791)
0927 07:01:03.751635 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.751749 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260927 07:01:03.758728 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.758173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:56159)
0927 07:01:03.758396 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.758536 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260927 07:01:03.763230 20528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:03.763502 20528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:03.764650 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.764128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.193:58419)
0927 07:01:03.764368 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.764469 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260927 07:01:03.765094 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.764525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.13.121.195:33149)
0927 07:01:03.764793 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:03.764884 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260927 07:01:03.771339 20528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c72c27da588488b9d00ed0d698884bc: Sending DeleteTablet for 3 replicas of tablet 1e9d27a9dc33440b9f57b44176610c9f
I20260927 07:01:03.772320 20833 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9d27a9dc33440b9f57b44176610c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:03 UTC) from {username='oryx', principal='oryx/127.13.121.253@KRBTEST.COM'} at 127.0.0.1:43202
I20260927 07:01:03.772302 20980 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9d27a9dc33440b9f57b44176610c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:03 UTC) from {username='oryx', principal='oryx/127.13.121.253@KRBTEST.COM'} at 127.0.0.1:50508
I20260927 07:01:03.772511 21121 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9d27a9dc33440b9f57b44176610c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:03 UTC) from {username='oryx', principal='oryx/127.13.121.253@KRBTEST.COM'} at 127.0.0.1:46548
I20260927 07:01:03.772996 21313 tablet_replica.cc:330] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: stopping tablet replica
I20260927 07:01:03.773169 21313 raft_consensus.cc:2229] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:01:03.773528 21313 raft_consensus.cc:2258] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:03.773737 21120 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9d27a9dc33440b9f57b44176610c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:03 UTC) from {username='oryx', principal='oryx/127.13.121.253@KRBTEST.COM'} at 127.0.0.1:46548
I20260927 07:01:03.774207 20833 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9d27a9dc33440b9f57b44176610c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:03 UTC) from {username='oryx', principal='oryx/127.13.121.253@KRBTEST.COM'} at 127.0.0.1:43202
I20260927 07:01:03.775019 21313 ts_tablet_manager.cc:1905] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:03.775907 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 20703
I20260927 07:01:03.782034 21313 ts_tablet_manager.cc:1918] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:03.782218 21313 log.cc:1196] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/1e9d27a9dc33440b9f57b44176610c9f
I20260927 07:01:03.782697 21313 ts_tablet_manager.cc:1939] T 1e9d27a9dc33440b9f57b44176610c9f P 6011aebe827d4d8cb9596a63325bc278: Deleting consensus metadata
I20260927 07:01:03.784555 20515 catalog_manager.cc:4841] TS 6011aebe827d4d8cb9596a63325bc278 (127.13.121.193:43887): tablet 1e9d27a9dc33440b9f57b44176610c9f (table test-table [id=d9ec2ecb4b9e40b180d3a22faa581986]) successfully deleted
I20260927 07:01:03.784900 21315 tablet_replica.cc:330] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: stopping tablet replica
I20260927 07:01:03.785130 21315 raft_consensus.cc:2229] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:03.785444 21315 raft_consensus.cc:2258] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 07:01:03.785728 20515 catalog_manager.cc:4808] TS 6011aebe827d4d8cb9596a63325bc278 (127.13.121.193:43887): delete failed for tablet 1e9d27a9dc33440b9f57b44176610c9f because the tablet was not found. No further retry: Not found: Tablet not found: 1e9d27a9dc33440b9f57b44176610c9f
I20260927 07:01:03.786124 21314 tablet_replica.cc:330] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: stopping tablet replica
I20260927 07:01:03.786299 21314 raft_consensus.cc:2229] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:03.786553 21314 raft_consensus.cc:2258] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:03.787858 21314 ts_tablet_manager.cc:1905] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:03.787953 21315 ts_tablet_manager.cc:1905] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:03.794509 21314 ts_tablet_manager.cc:1918] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:03.794659 21314 log.cc:1196] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/1e9d27a9dc33440b9f57b44176610c9f
I20260927 07:01:03.795091 21314 ts_tablet_manager.cc:1939] T 1e9d27a9dc33440b9f57b44176610c9f P ef99443da8a04883ab5bd0928f6ed985: Deleting consensus metadata
I20260927 07:01:03.795995 21315 ts_tablet_manager.cc:1918] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:03.796131 21315 log.cc:1196] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/1e9d27a9dc33440b9f57b44176610c9f
I20260927 07:01:03.796245 20514 catalog_manager.cc:4841] TS ef99443da8a04883ab5bd0928f6ed985 (127.13.121.194:36965): tablet 1e9d27a9dc33440b9f57b44176610c9f (table test-table [id=d9ec2ecb4b9e40b180d3a22faa581986]) successfully deleted
I20260927 07:01:03.796456 21315 ts_tablet_manager.cc:1939] T 1e9d27a9dc33440b9f57b44176610c9f P 6579c7d00e164f199ef48433840423d2: Deleting consensus metadata
I20260927 07:01:03.797401 20514 catalog_manager.cc:4841] TS 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693): tablet 1e9d27a9dc33440b9f57b44176610c9f (table test-table [id=d9ec2ecb4b9e40b180d3a22faa581986]) successfully deleted
W20260927 07:01:03.797695 20514 catalog_manager.cc:4808] TS 6579c7d00e164f199ef48433840423d2 (127.13.121.195:36693): delete failed for tablet 1e9d27a9dc33440b9f57b44176610c9f because the tablet was not found. No further retry: Not found: Tablet not found: 1e9d27a9dc33440b9f57b44176610c9f
I20260927 07:01:03.800216 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.777091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.13.121.193:43887)
0927 07:01:03.777266 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:03.777274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:03.787410 (+ 10136us) server_negotiation.cc:407] Connection header received
0927 07:01:03.787583 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:03.787588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:03.787676 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:03.787790 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:03.789001 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.790059 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.791242 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.791674 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.791787 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:03.794218 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:03.794233 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:03.794249 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:03.794288 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:03.797363 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:03.798164 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:03.798172 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:03.798174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:03.798305 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:03.798654 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:03.798659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:03.798661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:03.799037 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:03.799329 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:03.799678 (+ 349us) server_negotiation.cc:299] Negotiation successful
0927 07:01:03.799886 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 07:01:03.800809 20893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:03.919301 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 20703
I20260927 07:01:03.939749 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 20910
I20260927 07:01:03.952975 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:03.940421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.13.121.194:36965)
0927 07:01:03.940605 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:03.940610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:03.940629 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:01:03.940772 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:03.940776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:03.940845 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:03.940928 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:03.942141 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.942813 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.944020 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:03.944407 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:03.944509 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:03.947348 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:03.947365 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:03.947382 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:03.947421 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:03.950245 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:03.951053 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:03.951061 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:03.951063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:03.951246 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:03.951628 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:03.951632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:03.951634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:03.952082 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:03.952327 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:03.952551 (+ 224us) server_negotiation.cc:299] Negotiation successful
0927 07:01:03.952721 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:01:03.953732 21040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:04.045500 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 20910
I20260927 07:01:04.061616 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 21058
I20260927 07:01:04.073833 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:04.062300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46558 (local address 127.13.121.195:36693)
0927 07:01:04.062494 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:04.062499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:04.062519 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:01:04.062580 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:04.062583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:04.062658 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:04.062740 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:04.064119 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:04.064830 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:04.066015 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:04.066436 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:04.066534 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:04.068966 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:04.068980 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:04.068997 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:04.069033 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:04.071624 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:04.072203 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:04.072210 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:04.072212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:04.072327 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:04.072606 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:04.072610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:04.072611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:04.072946 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:04.073162 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:04.073377 (+ 215us) server_negotiation.cc:299] Negotiation successful
0927 07:01:04.073534 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 07:01:04.074440 21181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:04.112165 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:04.111449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35980)
0927 07:01:04.111874 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:04.111996 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":220,"threads_started":1}
I20260927 07:01:04.113479 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:04.113105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:44207 (local address 127.0.0.1:35986)
0927 07:01:04.113264 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:04.113349 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:44207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260927 07:01:04.171908 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 21058
I20260927 07:01:04.187196 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 20392
I20260927 07:01:04.188202 20559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:04.307633 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 20392
I20260927 07:01:04.325029 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 20492
I20260927 07:01:04.326061 20748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:04.438445 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 20492
2026-09-27T07:01:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11045 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:01:04 dist-test-slave-p6hz krb5kdc[21326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:01:04 dist-test-slave-p6hz krb5kdc[21326](info): set up 2 sockets
Sep 27 07:01:04 dist-test-slave-p6hz krb5kdc[21326](info): commencing operation
krb5kdc: starting...
W20260927 07:01:05.452049 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.007s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:01:05 dist-test-slave-p6hz krb5kdc[21326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:01:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:01:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.254.keytab.
Entry for principal oryx/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.254.keytab.
I20260927 07:01:05.668756 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:36709
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.254.keytab
--principal=oryx/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:36709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:06.003558 21339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:06.003865 21339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:06.003995 21339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:06.011458 21339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:01:06.011579 21339 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 07:01:06.011622 21339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:06.011706 21339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:06.011773 21339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:01:06.011857 21339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:01:06.022071 21339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:36709
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.254.keytab
--principal=oryx/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:36709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:06.023504 21339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:06.024830 21339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:06.036755 21347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:06.036685 21344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:06.036705 21345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:06.037842 21339 server_base.cc:1034] running on GCE node
I20260927 07:01:06.038647 21339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:06.040089 21339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:06.041301 21339 hybrid_clock.cc:648] HybridClock initialized: now 1790492466041214 us; error 60 us; skew 500 ppm
I20260927 07:01:06.041651 21339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:06 dist-test-slave-p6hz krb5kdc[21326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492466, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:06.048380 21339 init.cc:377] Logged in from keytab as oryx/127.13.121.254@KRBTEST.COM (short username oryx)
I20260927 07:01:06.049983 21339 webserver.cc:460] Webserver started at http://127.13.121.254:41101/ using document root <none> and password file <none>
I20260927 07:01:06.050686 21339 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:06.050774 21339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:06.051041 21339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:06.054112 21339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "63d5b3b7179547d593194a0575c81efb"
format_stamp: "Formatted at 2026-09-27 07:01:06 on dist-test-slave-p6hz"
server_key: "af7d6ed49ed822a792af6c3774944ee9"
server_key_iv: "5d4b36651f3847a9774433a21d553047"
server_key_version: "encryptionkey@0"
I20260927 07:01:06.055061 21339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "63d5b3b7179547d593194a0575c81efb"
format_stamp: "Formatted at 2026-09-27 07:01:06 on dist-test-slave-p6hz"
server_key: "af7d6ed49ed822a792af6c3774944ee9"
server_key_iv: "5d4b36651f3847a9774433a21d553047"
server_key_version: "encryptionkey@0"
I20260927 07:01:06.061389 21339 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 07:01:06.065464 21354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:06.066987 21339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:01:06.067225 21339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "63d5b3b7179547d593194a0575c81efb"
format_stamp: "Formatted at 2026-09-27 07:01:06 on dist-test-slave-p6hz"
server_key: "af7d6ed49ed822a792af6c3774944ee9"
server_key_iv: "5d4b36651f3847a9774433a21d553047"
server_key_version: "encryptionkey@0"
I20260927 07:01:06.067467 21339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:06.083205 21339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:06.087560 21339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:06.087821 21339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:06.109953 21339 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:36709
I20260927 07:01:06.109947 21415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:36709 every 8 connection(s)
I20260927 07:01:06.111327 21339 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:01:06.114964 21416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:06.120141 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 21339
I20260927 07:01:06.120340 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:01:06.120973 13799 external_mini_cluster.cc:1428] Setting key 855744feb4f2088db885461d5ebe64c3
I20260927 07:01:06.125986 21416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb: Bootstrap starting.
I20260927 07:01:06.129750 21416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:06.131103 21416 log.cc:824] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:06.134909 21416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb: No bootstrap required, opened a new log
Sep 27 07:01:06 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:01:06.144174 21416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d5b3b7179547d593194a0575c81efb" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36709 } }
I20260927 07:01:06.144688 21416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:06.144804 21416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d5b3b7179547d593194a0575c81efb, State: Initialized, Role: FOLLOWER
I20260927 07:01:06.145466 21416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d5b3b7179547d593194a0575c81efb" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36709 } }
I20260927 07:01:06.145710 21416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:01:06.145840 21416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:01:06.145999 21416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:06.146127 21419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:06.122970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.13.121.254:36709)
0927 07:01:06.123799 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:06.123813 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:06.123862 (+ 49us) server_negotiation.cc:407] Connection header received
0927 07:01:06.125693 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:06.125721 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:06.126196 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:06.126600 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:06.128460 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.129848 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.131712 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.132816 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.132984 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:06.137989 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:06.137999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:06.138020 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:06.138060 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:06.141192 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:06.142199 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:06.142208 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:06.142215 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:06.142463 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:06.142968 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:06.142973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:06.142975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:06.143715 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:06.143961 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:06.144489 (+ 528us) server_negotiation.cc:299] Negotiation successful
0927 07:01:06.144949 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":398,"threads_started":1}
I20260927 07:01:06.148423 21416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d5b3b7179547d593194a0575c81efb" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36709 } }
I20260927 07:01:06.148963 21416 leader_election.cc:302] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63d5b3b7179547d593194a0575c81efb; no voters:
I20260927 07:01:06.149613 21416 leader_election.cc:288] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:01:06.149740 21421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:06.150360 21421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [term 1 LEADER]: Becoming Leader. State: Replica: 63d5b3b7179547d593194a0575c81efb, State: Running, Role: LEADER
I20260927 07:01:06.150890 21421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d5b3b7179547d593194a0575c81efb" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36709 } }
I20260927 07:01:06.151628 21416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:01:06.155026 21423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63d5b3b7179547d593194a0575c81efb. Latest consensus state: current_term: 1 leader_uuid: "63d5b3b7179547d593194a0575c81efb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d5b3b7179547d593194a0575c81efb" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36709 } } }
I20260927 07:01:06.155521 21423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:06.156080 21422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63d5b3b7179547d593194a0575c81efb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d5b3b7179547d593194a0575c81efb" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36709 } } }
I20260927 07:01:06.156363 21422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:06.156822 21433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:01:06.161983 21433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:01:06.172688 21433 catalog_manager.cc:1347] Generated new cluster ID: 1b8c8068d7ca4204920705c87c781735
I20260927 07:01:06.172814 21433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:01:06.207446 21433 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:01:06.208936 21433 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:01:06.222277 21433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb: Generated new TSK 0
I20260927 07:01:06.223281 21433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab.
Entry for principal oryx/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab.
I20260927 07:01:06.287637 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36709
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab
--principal=oryx/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:06.637835 21442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:06.638140 21442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:06.638237 21442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:06.645574 21442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:06.645696 21442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:06.645779 21442 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:01:06.645857 21442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:01:06.656566 21442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab
--principal=oryx/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.121.254:36709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:06.657872 21442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:06.659386 21442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:06.673691 21447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:06.673717 21450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:06.673731 21448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:06.675336 21442 server_base.cc:1034] running on GCE node
I20260927 07:01:06.676110 21442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:06.677039 21442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:06.678290 21442 hybrid_clock.cc:648] HybridClock initialized: now 1790492466678207 us; error 57 us; skew 500 ppm
I20260927 07:01:06.678612 21442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:06 dist-test-slave-p6hz krb5kdc[21326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492466, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:06.681860 21442 init.cc:377] Logged in from keytab as oryx/127.13.121.193@KRBTEST.COM (short username oryx)
I20260927 07:01:06.683501 21442 webserver.cc:460] Webserver started at http://127.13.121.193:33233/ using document root <none> and password file <none>
I20260927 07:01:06.684207 21442 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:06.684301 21442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:06.684573 21442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:06.687629 21442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "0e06e761b24e4c679f7e2cf307d5e763"
format_stamp: "Formatted at 2026-09-27 07:01:06 on dist-test-slave-p6hz"
server_key: "24798b6a7b7d750d4844e79ce9677906"
server_key_iv: "9386864d59049d43ba14c746873a22bd"
server_key_version: "encryptionkey@0"
I20260927 07:01:06.688504 21442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "0e06e761b24e4c679f7e2cf307d5e763"
format_stamp: "Formatted at 2026-09-27 07:01:06 on dist-test-slave-p6hz"
server_key: "24798b6a7b7d750d4844e79ce9677906"
server_key_iv: "9386864d59049d43ba14c746873a22bd"
server_key_version: "encryptionkey@0"
I20260927 07:01:06.694212 21442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:01:06.698359 21457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:06.700101 21442 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 07:01:06.700281 21442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "0e06e761b24e4c679f7e2cf307d5e763"
format_stamp: "Formatted at 2026-09-27 07:01:06 on dist-test-slave-p6hz"
server_key: "24798b6a7b7d750d4844e79ce9677906"
server_key_iv: "9386864d59049d43ba14c746873a22bd"
server_key_version: "encryptionkey@0"
I20260927 07:01:06.700495 21442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:06.721374 21442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:06.725800 21442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:06.726049 21442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:06.728793 21442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:06.728890 21442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:06.729012 21442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:06.729067 21442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:01:06 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492466, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:01:06.753824 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:06.730408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:34559 (local address 127.13.121.254:36709)
0927 07:01:06.730875 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:06.730882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:06.731026 (+ 144us) server_negotiation.cc:407] Connection header received
0927 07:01:06.733566 (+ 2540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:06.733572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:06.733664 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:06.733776 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:06.736335 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.737281 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.739846 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.740514 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.740686 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:06.746341 (+ 5655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:06.746348 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:06.746351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:06.746497 (+ 146us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:06.749902 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:06.751056 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:06.751064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:06.751068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:06.751283 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:06.751897 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:06.751903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:06.751907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:06.752233 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:06.752374 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:06.753220 (+ 846us) server_negotiation.cc:299] Negotiation successful
0927 07:01:06.753463 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":640,"threads_started":1}
I20260927 07:01:06.754964 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:06.729961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36709 (local address 127.13.121.193:34559)
0927 07:01:06.730840 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:06.730894 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:06.733083 (+ 2189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:06.734064 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:06.734076 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:06.734890 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:06.736100 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:06.736137 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.737462 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.737468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:06.739643 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:06.739662 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.740819 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.740828 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:06.741142 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:06.742033 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:06.742079 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:06.745937 (+ 3858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:06.750112 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:06.750128 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:06.750155 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:06.750855 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:06.751449 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:06.751454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:06.751458 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:06.751732 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:06.752363 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:06.752373 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:06.752805 (+ 432us) client_negotiation.cc:769] Sending connection context
0927 07:01:06.753092 (+ 287us) client_negotiation.cc:240] Negotiation successful
0927 07:01:06.753713 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"spinlock_wait_cycles":18304,"thread_start_us":490,"threads_started":1}
I20260927 07:01:06.788412 21442 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:43443
I20260927 07:01:06.788424 21577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:43443 every 8 connection(s)
I20260927 07:01:06.789920 21442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:01:06.790191 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:06.766922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:49497 (local address 127.13.121.254:36709)
0927 07:01:06.767122 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:06.767158 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:06.775954 (+ 8796us) server_negotiation.cc:407] Connection header received
0927 07:01:06.776137 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:06.776143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:06.776218 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:06.776304 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:06.777291 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.778013 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.780308 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.780788 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.780892 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:06.783807 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:06.783814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:06.783819 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:06.783864 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:06.787093 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:06.788095 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:06.788099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:06.788102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:06.788240 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:06.788743 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:06.788747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:06.788749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:06.789045 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:06.789166 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:06.789672 (+ 506us) server_negotiation.cc:299] Negotiation successful
0927 07:01:06.789903 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:01:06.790364 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:06.766511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36709 (local address 127.13.121.193:49497)
0927 07:01:06.775776 (+ 9265us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:06.775801 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:06.776010 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:06.776351 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:06.776356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:06.776843 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:06.777133 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:06.777143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.778226 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.778234 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:06.780113 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:06.780140 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:06.781207 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:06.781217 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:06.781412 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:06.782264 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:06.782289 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:06.783552 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:06.787262 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:06.787272 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:06.787334 (+ 62us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:06.787939 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:06.788361 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:06.788367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:06.788371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:06.788601 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:06.789165 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:06.789170 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:06.789376 (+ 206us) client_negotiation.cc:769] Sending connection context
0927 07:01:06.789542 (+ 166us) client_negotiation.cc:240] Negotiation successful
0927 07:01:06.789872 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9077,"thread_start_us":8886,"threads_started":1}
I20260927 07:01:06.792320 21578 heartbeater.cc:347] Connected to a master server at 127.13.121.254:36709
I20260927 07:01:06.792547 21578 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:06.793237 21578 heartbeater.cc:511] Master 127.13.121.254:36709 requested a full tablet report, sending...
I20260927 07:01:06.795310 21371 ts_manager.cc:194] Registered new tserver with Master: 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443)
I20260927 07:01:06.796737 21371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:34559
I20260927 07:01:06.800354 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 21442
I20260927 07:01:06.800513 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:01:06.801168 13799 external_mini_cluster.cc:1428] Setting key 0e53a14051575f27626ecdb6c34d532c
WARNING: no policy specified for oryx/127.13.121.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab.
Entry for principal oryx/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab.
I20260927 07:01:06.855742 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36709
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab
--principal=oryx/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:07.191738 21582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:07.192065 21582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:07.192185 21582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:07.199678 21582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:07.199831 21582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:07.199965 21582 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:01:07.200083 21582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:01:07.210431 21582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab
--principal=oryx/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.121.254:36709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:07.211879 21582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:07.213281 21582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:07.227694 21587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:07.227728 21588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:07.227773 21590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:07.228853 21582 server_base.cc:1034] running on GCE node
I20260927 07:01:07.229509 21582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:07.230533 21582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:07.231758 21582 hybrid_clock.cc:648] HybridClock initialized: now 1790492467231681 us; error 55 us; skew 500 ppm
I20260927 07:01:07.232092 21582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:07 dist-test-slave-p6hz krb5kdc[21326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492467, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:07.235249 21582 init.cc:377] Logged in from keytab as oryx/127.13.121.194@KRBTEST.COM (short username oryx)
I20260927 07:01:07.236745 21582 webserver.cc:460] Webserver started at http://127.13.121.194:41761/ using document root <none> and password file <none>
I20260927 07:01:07.237426 21582 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:07.237511 21582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:07.237763 21582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:07.240877 21582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "a028898f5c4640c3afcdb1b13c45c410"
format_stamp: "Formatted at 2026-09-27 07:01:07 on dist-test-slave-p6hz"
server_key: "1fa6efbf00c647e18fcf701de20d1d0f"
server_key_iv: "b52154a85bea580356b08710d2ac4fde"
server_key_version: "encryptionkey@0"
I20260927 07:01:07.241686 21582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "a028898f5c4640c3afcdb1b13c45c410"
format_stamp: "Formatted at 2026-09-27 07:01:07 on dist-test-slave-p6hz"
server_key: "1fa6efbf00c647e18fcf701de20d1d0f"
server_key_iv: "b52154a85bea580356b08710d2ac4fde"
server_key_version: "encryptionkey@0"
I20260927 07:01:07.247309 21582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:01:07.252085 21597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:07.253903 21582 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260927 07:01:07.254091 21582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "a028898f5c4640c3afcdb1b13c45c410"
format_stamp: "Formatted at 2026-09-27 07:01:07 on dist-test-slave-p6hz"
server_key: "1fa6efbf00c647e18fcf701de20d1d0f"
server_key_iv: "b52154a85bea580356b08710d2ac4fde"
server_key_version: "encryptionkey@0"
I20260927 07:01:07.254328 21582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:07.293620 21582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:07.298210 21582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:07.298579 21582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:07.301337 21582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:07.301445 21582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:07.301563 21582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:07.301613 21582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:01:07 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492467, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:01:07.327437 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.303053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:42581 (local address 127.13.121.254:36709)
0927 07:01:07.303678 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:07.303684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:07.304736 (+ 1052us) server_negotiation.cc:407] Connection header received
0927 07:01:07.307531 (+ 2795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:07.307539 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:07.307647 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:07.307775 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:07.309895 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.310645 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.312907 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.313463 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.313604 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.319252 (+ 5648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:07.319261 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:07.319266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:07.319316 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:07.322959 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.324086 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.324092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.324096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.324267 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.324982 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.324988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.324992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.325360 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:07.325544 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:07.326732 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0927 07:01:07.327003 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":386,"threads_started":1}
I20260927 07:01:07.329151 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.303602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36709 (local address 127.13.121.194:42581)
0927 07:01:07.304563 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:07.304603 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:07.307107 (+ 2504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:07.308103 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:07.308120 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:07.308881 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:07.309669 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.309689 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.310809 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.310813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:07.312743 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.312758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.313753 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.313761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:07.314059 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.314972 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:07.315022 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:07.318882 (+ 3860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:07.323184 (+ 4302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:07.323196 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:07.323221 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:07.323916 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:07.324433 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:07.324439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:07.324444 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:07.324787 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:07.325521 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:07.325532 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:07.326167 (+ 635us) client_negotiation.cc:769] Sending connection context
0927 07:01:07.326578 (+ 411us) client_negotiation.cc:240] Negotiation successful
0927 07:01:07.327287 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":422,"threads_started":1}
I20260927 07:01:07.342335 21582 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:42687
I20260927 07:01:07.342347 21716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:42687 every 8 connection(s)
I20260927 07:01:07.344098 21582 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:01:07.346300 21718 heartbeater.cc:347] Connected to a master server at 127.13.121.254:36709
I20260927 07:01:07.346901 21718 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:07.347859 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 21582
I20260927 07:01:07.347865 21718 heartbeater.cc:511] Master 127.13.121.254:36709 requested a full tablet report, sending...
I20260927 07:01:07.348016 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:01:07.348589 13799 external_mini_cluster.cc:1428] Setting key 358cc5952aec6dcba5e55a37c8273725
I20260927 07:01:07.349318 21371 ts_manager.cc:194] Registered new tserver with Master: a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194:42687)
I20260927 07:01:07.350482 21371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:42581
I20260927 07:01:07.358783 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.341936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:43133 (local address 127.13.121.254:36709)
0927 07:01:07.342118 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:07.342125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:07.343705 (+ 1580us) server_negotiation.cc:407] Connection header received
0927 07:01:07.343878 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:07.343884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:07.343963 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:07.344056 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:07.345258 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.346180 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.348118 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.348778 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.348939 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.351085 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:07.351090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:07.351094 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:07.351122 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:07.354485 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.356068 (+ 1583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.356075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.356080 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.356232 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.357012 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.357019 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.357023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.357614 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:07.357799 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:07.358273 (+ 474us) server_negotiation.cc:299] Negotiation successful
0927 07:01:07.358524 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 07:01:07.358922 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.341967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36709 (local address 127.13.121.194:43133)
0927 07:01:07.343578 (+ 1611us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:07.343595 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:07.343754 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:07.344104 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:07.344109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:07.344644 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:07.345056 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.345070 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.346370 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.346376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:07.347893 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.347910 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.348970 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.348987 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:07.349152 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.349903 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:07.349927 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:07.350890 (+ 963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:07.354669 (+ 3779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:07.354676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:07.354680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:07.355911 (+ 1231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:07.356369 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:07.356375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:07.356379 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:07.356861 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:07.357750 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:07.357756 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:07.357964 (+ 208us) client_negotiation.cc:769] Sending connection context
0927 07:01:07.358108 (+ 144us) client_negotiation.cc:240] Negotiation successful
0927 07:01:07.358485 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1474,"thread_start_us":1028,"threads_started":1}
WARNING: no policy specified for oryx/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab.
Entry for principal oryx/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab.
I20260927 07:01:07.403201 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36709
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab
--principal=oryx/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:07.733054 21722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:07.733343 21722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:07.733443 21722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:07.740764 21722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:07.740902 21722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:07.740985 21722 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 07:01:07.741061 21722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:01:07.751260 21722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab
--principal=oryx/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.121.254:36709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:07.752629 21722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:07.754065 21722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:07.767686 21730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:07.767711 21727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:07.767686 21728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:07.768707 21722 server_base.cc:1034] running on GCE node
I20260927 07:01:07.769362 21722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:07.770790 21722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:07.772028 21722 hybrid_clock.cc:648] HybridClock initialized: now 1790492467771953 us; error 54 us; skew 500 ppm
I20260927 07:01:07.772349 21722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:07 dist-test-slave-p6hz krb5kdc[21326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492467, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:07.775609 21722 init.cc:377] Logged in from keytab as oryx/127.13.121.195@KRBTEST.COM (short username oryx)
I20260927 07:01:07.777329 21722 webserver.cc:460] Webserver started at http://127.13.121.195:34791/ using document root <none> and password file <none>
I20260927 07:01:07.778069 21722 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:07.778159 21722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:07.778450 21722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:07.781656 21722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "260e9d6fc9284eb69821e8194f29dbb9"
format_stamp: "Formatted at 2026-09-27 07:01:07 on dist-test-slave-p6hz"
server_key: "791a8492da6a6d3953d4bcff70fae12d"
server_key_iv: "fdc502964e745d576b8e712cd2e4b5f4"
server_key_version: "encryptionkey@0"
I20260927 07:01:07.782577 21722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "260e9d6fc9284eb69821e8194f29dbb9"
format_stamp: "Formatted at 2026-09-27 07:01:07 on dist-test-slave-p6hz"
server_key: "791a8492da6a6d3953d4bcff70fae12d"
server_key_iv: "fdc502964e745d576b8e712cd2e4b5f4"
server_key_version: "encryptionkey@0"
I20260927 07:01:07.788340 21722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:01:07.792919 21737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:07.794368 21722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:01:07.794519 21722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "260e9d6fc9284eb69821e8194f29dbb9"
format_stamp: "Formatted at 2026-09-27 07:01:07 on dist-test-slave-p6hz"
server_key: "791a8492da6a6d3953d4bcff70fae12d"
server_key_iv: "fdc502964e745d576b8e712cd2e4b5f4"
server_key_version: "encryptionkey@0"
I20260927 07:01:07.794747 21722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:07.799703 21578 heartbeater.cc:503] Master 127.13.121.254:36709 was elected leader, sending a full tablet report...
I20260927 07:01:07.832602 21722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:07.836884 21722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:07.837127 21722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:07.839701 21722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:07.839799 21722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:07.839926 21722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:07.839999 21722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:01:07 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492467, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:01:07.862908 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.840482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:44013 (local address 127.13.121.254:36709)
0927 07:01:07.840676 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:07.840682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:07.842160 (+ 1478us) server_negotiation.cc:407] Connection header received
0927 07:01:07.844393 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:07.844400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:07.844500 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:07.844608 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:07.846748 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.847920 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.850255 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.850813 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.850945 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.855653 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:07.855662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:07.855667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:07.855715 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:07.858915 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.860085 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.860092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.860095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.860249 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.860920 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.860925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.860927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.861230 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:07.861358 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:07.862429 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0927 07:01:07.862620 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:01:07.864719 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.841177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36709 (local address 127.13.121.195:44013)
0927 07:01:07.841991 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:07.842037 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:07.844042 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:07.844948 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:07.844960 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:07.845681 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:07.846490 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.846532 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.848068 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.848072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:07.850072 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.850090 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.851107 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.851116 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:07.851352 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.852067 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:07.852101 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:07.855276 (+ 3175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:07.859100 (+ 3824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:07.859113 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:07.859163 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:07.859925 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:07.860375 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:07.860381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:07.860385 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:07.860722 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:07.861353 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:07.861363 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:07.861934 (+ 571us) client_negotiation.cc:769] Sending connection context
0927 07:01:07.862269 (+ 335us) client_negotiation.cc:240] Negotiation successful
0927 07:01:07.862916 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":359,"threads_started":1}
I20260927 07:01:07.886596 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.871253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:47623 (local address 127.13.121.254:36709)
0927 07:01:07.871484 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:07.871492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:07.872318 (+ 826us) server_negotiation.cc:407] Connection header received
0927 07:01:07.872629 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:07.872635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:07.872700 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:07.872791 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:07.874238 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.875332 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.876814 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.877513 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.877661 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.880352 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:07.880361 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:07.880366 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:07.880416 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:07.883757 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.884615 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.884619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.884622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.884749 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.885232 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.885237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.885240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.885544 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:07.885674 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:07.886147 (+ 473us) server_negotiation.cc:299] Negotiation successful
0927 07:01:07.886333 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:01:07.886977 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.871288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36709 (local address 127.13.121.195:47623)
0927 07:01:07.872176 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:07.872196 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:07.872478 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:07.872997 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:07.873012 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:07.873649 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:07.874088 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.874103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.875479 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.875484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:07.876646 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.876668 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.877740 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.877747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:07.877973 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.878736 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:07.878762 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:07.880111 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:07.883956 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:07.883963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:07.883967 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:07.884460 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:07.884868 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:07.884874 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:07.884879 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:07.885065 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:07.885657 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:07.885662 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:07.885896 (+ 234us) client_negotiation.cc:769] Sending connection context
0927 07:01:07.886055 (+ 159us) client_negotiation.cc:240] Negotiation successful
0927 07:01:07.886472 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":579,"threads_started":1}
I20260927 07:01:07.890926 21722 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:35815
I20260927 07:01:07.890985 21856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:35815 every 8 connection(s)
I20260927 07:01:07.892175 21722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:01:07.894244 21857 heartbeater.cc:347] Connected to a master server at 127.13.121.254:36709
I20260927 07:01:07.894466 21857 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:07.894649 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 21722
I20260927 07:01:07.894786 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:01:07.895190 21857 heartbeater.cc:511] Master 127.13.121.254:36709 requested a full tablet report, sending...
I20260927 07:01:07.895268 13799 external_mini_cluster.cc:1428] Setting key 5330aeb8f040471379fe96d55ad0cb07
I20260927 07:01:07.896665 21370 ts_manager.cc:194] Registered new tserver with Master: 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815)
I20260927 07:01:07.897610 21370 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:44013
I20260927 07:01:07.898135 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 07:01:07 dist-test-slave-p6hz krb5kdc[21326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492467, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 07:01:07 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:07 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 07:01:07.945147 21606 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:07.934771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53142 (local address 127.13.121.254:36709)
0927 07:01:07.934928 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:07.934932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:07.935262 (+ 330us) server_negotiation.cc:407] Connection header received
0927 07:01:07.935393 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:07.935398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:07.935464 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:07.935554 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:07.936874 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.937613 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.939049 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.939549 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.939652 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.944926 (+ 5274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53142: BlockingRecv error: recv got EOF from 127.0.0.1:53142 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 27 07:01:07 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492467, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:01:07.962105 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.948203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.13.121.254:36709)
0927 07:01:07.948528 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:07.948536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:07.948670 (+ 134us) server_negotiation.cc:407] Connection header received
0927 07:01:07.948806 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:07.948810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:07.948889 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:07.948983 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:07.950222 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.950966 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.952308 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.953049 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.953159 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:07.957090 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:07.957096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:07.957100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:07.957140 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:07.959707 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.960410 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.960413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.960416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.960535 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:07.960854 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:07.960858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:07.960860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:07.961123 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:07.961311 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:07.961641 (+ 330us) server_negotiation.cc:299] Negotiation successful
0927 07:01:07.961804 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260927 07:01:07.966557 21370 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:01:07.973349 21370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:01:08.004853 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.986172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54110 (local address 127.13.121.194:42687)
0927 07:01:07.994431 (+ 8259us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:07.994443 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:07.995256 (+ 813us) server_negotiation.cc:407] Connection header received
0927 07:01:07.995581 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:07.995588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:07.995792 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:07.995998 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:07.996901 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.998265 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.001532 (+ 3267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.003005 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.003186 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.003380 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.004261 (+ 881us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.004509 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8056,"thread_start_us":7921,"threads_started":1}
I20260927 07:01:08.006160 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.985881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42687 (local address 127.0.0.1:54110)
0927 07:01:07.995054 (+ 9173us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:07.995100 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:07.995319 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:07.996242 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:07.996250 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:07.996282 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:07.996688 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.996706 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.998460 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.998466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.001301 (+ 2835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.001321 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.003695 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.003703 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.003946 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.004007 (+ 61us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.005464 (+ 1457us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.005704 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8911,"mutex_wait_us":36,"thread_start_us":8770,"threads_started":1}
I20260927 07:01:08.006292 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.986002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59582 (local address 127.13.121.195:35815)
0927 07:01:07.992816 (+ 6814us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:07.992825 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:07.996031 (+ 3206us) server_negotiation.cc:407] Connection header received
0927 07:01:07.996141 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:07.996150 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:07.996413 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:07.996616 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:07.997677 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.999513 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.002517 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.004418 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.004642 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.004829 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.005636 (+ 807us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.005839 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6626,"thread_start_us":6464,"threads_started":1}
I20260927 07:01:08.006994 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.986110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:35815 (local address 127.0.0.1:59582)
0927 07:01:07.995765 (+ 9655us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:07.995789 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:07.995905 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:07.996935 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:07.996940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:07.996968 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:07.997409 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:07.997421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:07.999701 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:07.999707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.002249 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.002270 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.005043 (+ 2773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.005050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.005256 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.005310 (+ 54us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.005413 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.006603 (+ 1190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9520,"mutex_wait_us":937,"thread_start_us":9357,"threads_started":1}
I20260927 07:01:08.011263 21648 tablet_service.cc:1467] Processing CreateTablet for tablet 8cc92057bd4745ff9106503f1744c203 (DEFAULT_TABLE table=test-table [id=63cd79aec2334d8a954db3ef611f9ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.012513 21648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cc92057bd4745ff9106503f1744c203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.012303 21785 tablet_service.cc:1467] Processing CreateTablet for tablet 8cc92057bd4745ff9106503f1744c203 (DEFAULT_TABLE table=test-table [id=63cd79aec2334d8a954db3ef611f9ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.013710 21785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cc92057bd4745ff9106503f1744c203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.015848 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.995916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43443 (local address 127.0.0.1:35088)
0927 07:01:08.007520 (+ 11604us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:08.007542 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:08.007670 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:08.009073 (+ 1403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:08.009078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:08.009106 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:08.009290 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.009301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.011267 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.011273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.013218 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.013237 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.015058 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.015065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.015282 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.015319 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.015422 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.015567 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11474,"thread_start_us":12959,"threads_started":1}
I20260927 07:01:08.015928 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:07.996202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35088 (local address 127.13.121.193:43443)
0927 07:01:08.008408 (+ 12206us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.008417 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.008448 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:01:08.008588 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.008596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.008861 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.009468 (+ 607us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:08.009678 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.011100 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.013418 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.014841 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.015035 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.015230 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.015471 (+ 241us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.015646 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12048,"thread_start_us":11894,"threads_started":1}
I20260927 07:01:08.020174 21508 tablet_service.cc:1467] Processing CreateTablet for tablet 8cc92057bd4745ff9106503f1744c203 (DEFAULT_TABLE table=test-table [id=63cd79aec2334d8a954db3ef611f9ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.021708 21508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cc92057bd4745ff9106503f1744c203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.024328 21881 tablet_bootstrap.cc:492] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: Bootstrap starting.
I20260927 07:01:08.027449 21881 tablet_bootstrap.cc:654] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.028508 21882 tablet_bootstrap.cc:492] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: Bootstrap starting.
I20260927 07:01:08.028697 21881 log.cc:824] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:08.032068 21882 tablet_bootstrap.cc:654] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.032399 21881 tablet_bootstrap.cc:492] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: No bootstrap required, opened a new log
I20260927 07:01:08.032707 21881 ts_tablet_manager.cc:1397] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260927 07:01:08.033340 21882 log.cc:824] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:08.036444 21882 tablet_bootstrap.cc:492] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: No bootstrap required, opened a new log
I20260927 07:01:08.036718 21882 ts_tablet_manager.cc:1397] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260927 07:01:08.039857 21885 tablet_bootstrap.cc:492] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: Bootstrap starting.
I20260927 07:01:08.042997 21885 tablet_bootstrap.cc:654] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.042727 21881 raft_consensus.cc:348] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } }
I20260927 07:01:08.043249 21881 raft_consensus.cc:374] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.043351 21881 raft_consensus.cc:729] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a028898f5c4640c3afcdb1b13c45c410, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.044025 21881 consensus_queue.cc:260] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } }
I20260927 07:01:08.044222 21885 log.cc:824] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:08.045214 21718 heartbeater.cc:503] Master 127.13.121.254:36709 was elected leader, sending a full tablet report...
I20260927 07:01:08.045871 21881 ts_tablet_manager.cc:1428] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260927 07:01:08.047868 21882 raft_consensus.cc:348] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } }
I20260927 07:01:08.048244 21882 raft_consensus.cc:374] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.048345 21882 raft_consensus.cc:729] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260e9d6fc9284eb69821e8194f29dbb9, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.048924 21882 consensus_queue.cc:260] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } }
I20260927 07:01:08.049453 21886 raft_consensus.cc:482] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:01:08.049780 21886 raft_consensus.cc:504] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } }
I20260927 07:01:08.050546 21885 tablet_bootstrap.cc:492] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: No bootstrap required, opened a new log
I20260927 07:01:08.050843 21885 ts_tablet_manager.cc:1397] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20260927 07:01:08.055774 21886 leader_election.cc:288] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815), 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443)
I20260927 07:01:08.062945 21885 raft_consensus.cc:348] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } }
I20260927 07:01:08.063725 21885 raft_consensus.cc:374] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.063868 21885 raft_consensus.cc:729] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e06e761b24e4c679f7e2cf307d5e763, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.065316 21857 heartbeater.cc:503] Master 127.13.121.254:36709 was elected leader, sending a full tablet report...
I20260927 07:01:08.065944 21882 ts_tablet_manager.cc:1428] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: Time spent starting tablet: real 0.029s user 0.021s sys 0.007s
I20260927 07:01:08.067114 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.052256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:60343 (local address 127.13.121.195:35815)
0927 07:01:08.052626 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.052633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.056861 (+ 4228us) server_negotiation.cc:407] Connection header received
0927 07:01:08.057261 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.057268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.057357 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.057468 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:08.058575 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.059917 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.063084 (+ 3167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.064684 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.064854 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.064931 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.066702 (+ 1771us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.066847 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:01:08.067590 21885 consensus_queue.cc:260] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } }
I20260927 07:01:08.068212 21889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.051898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:35815 (local address 127.13.121.194:60343)
0927 07:01:08.056720 (+ 4822us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:08.056746 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:08.057018 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:08.057772 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:08.057779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:08.057810 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:08.058325 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.058340 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.060288 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.060294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.062793 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.062817 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.066229 (+ 3412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.066242 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.066462 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.066502 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.067489 (+ 987us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.067660 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4602,"thread_start_us":4409,"threads_started":1}
I20260927 07:01:08.069275 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cc92057bd4745ff9106503f1744c203" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9" is_pre_election: true
I20260927 07:01:08.069806 21807 raft_consensus.cc:2454] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 0.
I20260927 07:01:08.069998 21885 ts_tablet_manager.cc:1428] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
I20260927 07:01:08.070644 21599 leader_election.cc:302] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 260e9d6fc9284eb69821e8194f29dbb9, a028898f5c4640c3afcdb1b13c45c410; no voters:
I20260927 07:01:08.071085 21886 raft_consensus.cc:2781] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:01:08.071272 21886 raft_consensus.cc:482] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:01:08.071391 21886 raft_consensus.cc:3037] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.074733 21886 raft_consensus.cc:504] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } }
I20260927 07:01:08.075007 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.057057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:35445 (local address 127.13.121.193:43443)
0927 07:01:08.057268 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.057277 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.067859 (+ 10582us) server_negotiation.cc:407] Connection header received
0927 07:01:08.069100 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.069106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.069180 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.069237 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:08.069979 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.070944 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.073186 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.074156 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.074296 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.074369 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.074655 (+ 286us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.074775 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 07:01:08.075454 21890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.057417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43443 (local address 127.13.121.194:35445)
0927 07:01:08.067748 (+ 10331us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:08.067768 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:08.068930 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:08.069437 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:08.069442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:08.069468 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:08.069801 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.069813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.071320 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.071324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.073024 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.073041 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.074358 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.074364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.074513 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.074537 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.074617 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.074761 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10177,"thread_start_us":10439,"threads_started":1}
I20260927 07:01:08.075546 21886 leader_election.cc:288] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 election: Requested vote from peers 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815), 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443)
I20260927 07:01:08.075938 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cc92057bd4745ff9106503f1744c203" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9"
I20260927 07:01:08.076134 21807 raft_consensus.cc:3037] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.076674 21528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cc92057bd4745ff9106503f1744c203" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e06e761b24e4c679f7e2cf307d5e763" is_pre_election: true
I20260927 07:01:08.076699 21527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cc92057bd4745ff9106503f1744c203" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e06e761b24e4c679f7e2cf307d5e763"
I20260927 07:01:08.077154 21528 raft_consensus.cc:2454] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 0.
I20260927 07:01:08.078720 21807 raft_consensus.cc:2454] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 1.
I20260927 07:01:08.079185 21599 leader_election.cc:302] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 260e9d6fc9284eb69821e8194f29dbb9, a028898f5c4640c3afcdb1b13c45c410; no voters: 0e06e761b24e4c679f7e2cf307d5e763
I20260927 07:01:08.079492 21886 raft_consensus.cc:2781] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:08.079933 21886 raft_consensus.cc:686] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 1 LEADER]: Becoming Leader. State: Replica: a028898f5c4640c3afcdb1b13c45c410, State: Running, Role: LEADER
I20260927 07:01:08.080469 21886 consensus_queue.cc:237] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } }
I20260927 07:01:08.085291 21370 catalog_manager.cc:5495] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 reported cstate change: term changed from 0 to 1, leader changed from <none> to a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194). New cstate: current_term: 1 leader_uuid: "a028898f5c4640c3afcdb1b13c45c410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } health_report { overall_health: UNKNOWN } } }
W20260927 07:01:08.097150 21719 tablet.cc:2369] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:01:08.124145 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.117897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.13.121.194:42687)
0927 07:01:08.118090 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.118096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.118195 (+ 99us) server_negotiation.cc:407] Connection header received
0927 07:01:08.118382 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.118387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.118486 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.118653 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:08.119055 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.120160 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.121469 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.122128 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.122256 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.122762 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:08.123518 (+ 756us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:08.123656 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.123762 (+ 106us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.123852 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:01:08.132768 21527 raft_consensus.cc:3037] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.133015 21807 raft_consensus.cc:1261] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Refusing update from remote peer a028898f5c4640c3afcdb1b13c45c410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:08.133879 21886 consensus_queue.cc:1035] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:08.135509 21527 raft_consensus.cc:1261] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Refusing update from remote peer a028898f5c4640c3afcdb1b13c45c410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:08.136263 21886 consensus_queue.cc:1035] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:08.143167 21895 mvcc.cc:205] Tried to move back new op lower bound from 7333857149461839872 to 7333857149261586432. Current Snapshot: MvccSnapshot[applied={T|T < 7333857149461839872}]
I20260927 07:01:08.145619 21896 mvcc.cc:205] Tried to move back new op lower bound from 7333857149461839872 to 7333857149261586432. Current Snapshot: MvccSnapshot[applied={T|T < 7333857149461839872}]
I20260927 07:01:08.154989 21898 mvcc.cc:205] Tried to move back new op lower bound from 7333857149461839872 to 7333857149261586432. Current Snapshot: MvccSnapshot[applied={T|T < 7333857149461839872}]
I20260927 07:01:08.163703 21370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:08.164003 21370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:08.168367 21370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb: Sending DeleteTablet for 3 replicas of tablet 8cc92057bd4745ff9106503f1744c203
I20260927 07:01:08.169448 21648 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc92057bd4745ff9106503f1744c203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:54110
I20260927 07:01:08.169521 21508 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc92057bd4745ff9106503f1744c203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35088
I20260927 07:01:08.169850 21507 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc92057bd4745ff9106503f1744c203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35088
I20260927 07:01:08.170364 21785 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc92057bd4745ff9106503f1744c203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:59582
I20260927 07:01:08.170953 21370 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:01:08.171797 21370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:01:08.180142 21647 tablet_service.cc:1467] Processing CreateTablet for tablet d42e20d9d7db47cd866eb3becb70f36e (DEFAULT_TABLE table=test-table [id=0d4c20dd8db84f97a674e08f97d6a90b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.180622 21507 tablet_service.cc:1467] Processing CreateTablet for tablet d42e20d9d7db47cd866eb3becb70f36e (DEFAULT_TABLE table=test-table [id=0d4c20dd8db84f97a674e08f97d6a90b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.180768 21647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42e20d9d7db47cd866eb3becb70f36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.180677 21787 tablet_service.cc:1467] Processing CreateTablet for tablet d42e20d9d7db47cd866eb3becb70f36e (DEFAULT_TABLE table=test-table [id=0d4c20dd8db84f97a674e08f97d6a90b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.181176 21787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42e20d9d7db47cd866eb3becb70f36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.181200 21507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42e20d9d7db47cd866eb3becb70f36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.188031 21885 tablet_bootstrap.cc:492] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: Bootstrap starting.
I20260927 07:01:08.188210 21881 tablet_bootstrap.cc:492] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: Bootstrap starting.
I20260927 07:01:08.189850 21885 tablet_bootstrap.cc:654] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.189881 21881 tablet_bootstrap.cc:654] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.190336 21907 tablet_replica.cc:330] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: stopping tablet replica
I20260927 07:01:08.190783 21907 raft_consensus.cc:2229] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:08.191424 21907 raft_consensus.cc:2258] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:08.192230 21905 tablet_replica.cc:330] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: stopping tablet replica
I20260927 07:01:08.192660 21905 raft_consensus.cc:2229] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:01:08.193391 21905 raft_consensus.cc:2258] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:08.194190 21906 tablet_replica.cc:330] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: stopping tablet replica
I20260927 07:01:08.194550 21882 tablet_bootstrap.cc:492] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: Bootstrap starting.
I20260927 07:01:08.194604 21906 raft_consensus.cc:2229] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:08.195111 21906 raft_consensus.cc:2258] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:08.196281 21882 tablet_bootstrap.cc:654] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.197552 21907 ts_tablet_manager.cc:1905] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:08.199427 21885 tablet_bootstrap.cc:492] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: No bootstrap required, opened a new log
I20260927 07:01:08.199589 21885 ts_tablet_manager.cc:1397] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.001s
I20260927 07:01:08.200436 21885 raft_consensus.cc:348] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.200695 21885 raft_consensus.cc:374] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.200786 21885 raft_consensus.cc:729] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e06e761b24e4c679f7e2cf307d5e763, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.201058 21885 consensus_queue.cc:260] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.201649 21885 ts_tablet_manager.cc:1428] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260927 07:01:08.202224 21881 tablet_bootstrap.cc:492] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: No bootstrap required, opened a new log
I20260927 07:01:08.202436 21881 ts_tablet_manager.cc:1397] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.009s
I20260927 07:01:08.203333 21906 ts_tablet_manager.cc:1905] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:08.203488 21881 raft_consensus.cc:348] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.203801 21881 raft_consensus.cc:374] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.203943 21881 raft_consensus.cc:729] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a028898f5c4640c3afcdb1b13c45c410, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.204433 21881 consensus_queue.cc:260] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.205181 21881 ts_tablet_manager.cc:1428] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 07:01:08.214964 21907 ts_tablet_manager.cc:1918] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:08.216216 21905 ts_tablet_manager.cc:1905] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:08.217758 21882 tablet_bootstrap.cc:492] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: No bootstrap required, opened a new log
I20260927 07:01:08.217911 21882 ts_tablet_manager.cc:1397] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.000s
I20260927 07:01:08.218707 21882 raft_consensus.cc:348] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.218937 21882 raft_consensus.cc:374] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.219033 21882 raft_consensus.cc:729] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260e9d6fc9284eb69821e8194f29dbb9, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.220678 21882 consensus_queue.cc:260] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.222370 21906 ts_tablet_manager.cc:1918] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:08.222582 21906 log.cc:1196] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/8cc92057bd4745ff9106503f1744c203
I20260927 07:01:08.223229 21905 ts_tablet_manager.cc:1918] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:08.223404 21905 log.cc:1196] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/8cc92057bd4745ff9106503f1744c203
I20260927 07:01:08.223600 21882 ts_tablet_manager.cc:1428] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260927 07:01:08.223870 21906 ts_tablet_manager.cc:1939] T 8cc92057bd4745ff9106503f1744c203 P 0e06e761b24e4c679f7e2cf307d5e763: Deleting consensus metadata
I20260927 07:01:08.223917 21905 ts_tablet_manager.cc:1939] T 8cc92057bd4745ff9106503f1744c203 P a028898f5c4640c3afcdb1b13c45c410: Deleting consensus metadata
I20260927 07:01:08.225700 21358 catalog_manager.cc:4841] TS a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194:42687): tablet 8cc92057bd4745ff9106503f1744c203 (table test-table [id=63cd79aec2334d8a954db3ef611f9ad2]) successfully deleted
I20260927 07:01:08.225942 21356 catalog_manager.cc:4841] TS 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443): tablet 8cc92057bd4745ff9106503f1744c203 (table test-table [id=63cd79aec2334d8a954db3ef611f9ad2]) successfully deleted
I20260927 07:01:08.226136 21907 log.cc:1196] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/8cc92057bd4745ff9106503f1744c203
W20260927 07:01:08.226540 21356 catalog_manager.cc:4808] TS 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443): delete failed for tablet 8cc92057bd4745ff9106503f1744c203 because the tablet was not found. No further retry: Not found: Tablet not found: 8cc92057bd4745ff9106503f1744c203
I20260927 07:01:08.226665 21907 ts_tablet_manager.cc:1939] T 8cc92057bd4745ff9106503f1744c203 P 260e9d6fc9284eb69821e8194f29dbb9: Deleting consensus metadata
I20260927 07:01:08.228327 21356 catalog_manager.cc:4841] TS 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815): tablet 8cc92057bd4745ff9106503f1744c203 (table test-table [id=63cd79aec2334d8a954db3ef611f9ad2]) successfully deleted
I20260927 07:01:08.228873 21902 raft_consensus.cc:482] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:01:08.229105 21902 raft_consensus.cc:504] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.229851 21902 leader_election.cc:288] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443), 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815)
I20260927 07:01:08.230324 21527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42e20d9d7db47cd866eb3becb70f36e" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e06e761b24e4c679f7e2cf307d5e763" is_pre_election: true
I20260927 07:01:08.230392 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42e20d9d7db47cd866eb3becb70f36e" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9" is_pre_election: true
I20260927 07:01:08.230559 21527 raft_consensus.cc:2454] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 0.
I20260927 07:01:08.230625 21807 raft_consensus.cc:2454] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 0.
I20260927 07:01:08.230983 21599 leader_election.cc:302] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e06e761b24e4c679f7e2cf307d5e763, a028898f5c4640c3afcdb1b13c45c410; no voters:
I20260927 07:01:08.231278 21902 raft_consensus.cc:2781] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:01:08.231396 21902 raft_consensus.cc:482] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:01:08.231494 21902 raft_consensus.cc:3037] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.233731 21902 raft_consensus.cc:504] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.234249 21902 leader_election.cc:288] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 election: Requested vote from peers 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443), 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815)
I20260927 07:01:08.234835 21527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42e20d9d7db47cd866eb3becb70f36e" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e06e761b24e4c679f7e2cf307d5e763"
I20260927 07:01:08.234876 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42e20d9d7db47cd866eb3becb70f36e" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9"
I20260927 07:01:08.235051 21807 raft_consensus.cc:3037] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.235072 21527 raft_consensus.cc:3037] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.237964 21527 raft_consensus.cc:2454] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 1.
I20260927 07:01:08.237987 21807 raft_consensus.cc:2454] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 1.
I20260927 07:01:08.238358 21599 leader_election.cc:302] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e06e761b24e4c679f7e2cf307d5e763, a028898f5c4640c3afcdb1b13c45c410; no voters:
I20260927 07:01:08.238626 21902 raft_consensus.cc:2781] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:08.238791 21902 raft_consensus.cc:686] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 1 LEADER]: Becoming Leader. State: Replica: a028898f5c4640c3afcdb1b13c45c410, State: Running, Role: LEADER
I20260927 07:01:08.239106 21902 consensus_queue.cc:237] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.241835 21370 catalog_manager.cc:5495] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 reported cstate change: term changed from 0 to 1, leader changed from <none> to a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194). New cstate: current_term: 1 leader_uuid: "a028898f5c4640c3afcdb1b13c45c410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } health_report { overall_health: UNKNOWN } } }
Sep 27 07:01:08 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492466, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.254@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:01:08.283661 21917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.269786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36709 (local address 127.0.0.1:53154)
0927 07:01:08.270350 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:08.270367 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:08.270511 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:08.270795 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:08.270799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:08.271593 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:08.271848 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.271857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.272800 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.272806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.273734 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.273746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.274444 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.274451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.274581 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.275224 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:08.275248 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:08.277627 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:08.281225 (+ 3598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:08.281236 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:08.281242 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:08.281648 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:08.282017 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:08.282020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:08.282023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:08.282137 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:08.282712 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:08.282718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:08.282889 (+ 171us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.283060 (+ 171us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.283334 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":324,"threads_started":1}
I20260927 07:01:08.283901 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.269702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53154 (local address 127.13.121.254:36709)
0927 07:01:08.269883 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.269889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.270461 (+ 572us) server_negotiation.cc:407] Connection header received
0927 07:01:08.270619 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.270623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.270675 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.270751 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:08.271992 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.272670 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.273874 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.274276 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.274388 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.277837 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:08.277842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:08.277845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:08.277882 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:08.281081 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:08.281758 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:08.281764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:08.281768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:08.281914 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:08.282226 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:08.282231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:08.282235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:08.282575 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:08.282716 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.283211 (+ 495us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.283503 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":64}
I20260927 07:01:08.288625 21370 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:53154:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 07:01:08.289620 21370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 07:01:08.292760 21579 tablet.cc:2369] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:01:08.296762 21508 tablet_service.cc:1467] Processing CreateTablet for tablet f28b27ddc4724bf5804e1f392e011c9f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=53db3435a0114dc6ac6c9c11b9b10b5b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:01:08.296908 21648 tablet_service.cc:1467] Processing CreateTablet for tablet f28b27ddc4724bf5804e1f392e011c9f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=53db3435a0114dc6ac6c9c11b9b10b5b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:01:08.297039 21787 tablet_service.cc:1467] Processing CreateTablet for tablet f28b27ddc4724bf5804e1f392e011c9f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=53db3435a0114dc6ac6c9c11b9b10b5b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 07:01:08.297488 21787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f28b27ddc4724bf5804e1f392e011c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.297510 21648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f28b27ddc4724bf5804e1f392e011c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.297669 21508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f28b27ddc4724bf5804e1f392e011c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.304767 21882 tablet_bootstrap.cc:492] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: Bootstrap starting.
I20260927 07:01:08.304963 21881 tablet_bootstrap.cc:492] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: Bootstrap starting.
I20260927 07:01:08.304955 21885 tablet_bootstrap.cc:492] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: Bootstrap starting.
I20260927 07:01:08.306560 21882 tablet_bootstrap.cc:654] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.306560 21881 tablet_bootstrap.cc:654] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.306650 21885 tablet_bootstrap.cc:654] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.315317 21885 tablet_bootstrap.cc:492] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: No bootstrap required, opened a new log
I20260927 07:01:08.315582 21885 ts_tablet_manager.cc:1397] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260927 07:01:08.316377 21885 raft_consensus.cc:348] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.316699 21885 raft_consensus.cc:374] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.316813 21885 raft_consensus.cc:729] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e06e761b24e4c679f7e2cf307d5e763, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.317147 21882 tablet_bootstrap.cc:492] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: No bootstrap required, opened a new log
I20260927 07:01:08.317126 21885 consensus_queue.cc:260] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.317358 21882 ts_tablet_manager.cc:1397] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260927 07:01:08.317693 21891 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.318039 21891 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:01:08.318115 21882 raft_consensus.cc:348] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.318351 21882 raft_consensus.cc:374] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.318423 21882 raft_consensus.cc:729] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260e9d6fc9284eb69821e8194f29dbb9, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.318681 21882 consensus_queue.cc:260] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.319264 21885 ts_tablet_manager.cc:1428] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260927 07:01:08.319100 21888 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.320261 21888 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:01:08.320627 21882 ts_tablet_manager.cc:1428] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 07:01:08.322225 21881 tablet_bootstrap.cc:492] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: No bootstrap required, opened a new log
I20260927 07:01:08.322463 21881 ts_tablet_manager.cc:1397] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.004s
I20260927 07:01:08.323334 21881 raft_consensus.cc:348] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.323619 21881 raft_consensus.cc:374] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.323705 21881 raft_consensus.cc:729] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a028898f5c4640c3afcdb1b13c45c410, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.323937 21881 consensus_queue.cc:260] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.324548 21881 ts_tablet_manager.cc:1428] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260927 07:01:08.324421 21902 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.324728 21902 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: This TxnStatusTablet replica's current role is: FOLLOWER
W20260927 07:01:08.394630 21858 tablet.cc:2369] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:01:08.518676 21891 raft_consensus.cc:482] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:01:08.518898 21891 raft_consensus.cc:504] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.520473 21891 leader_election.cc:288] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194:42687), 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815)
I20260927 07:01:08.527802 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.520870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:47521 (local address 127.13.121.195:35815)
0927 07:01:08.521238 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.521245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.521598 (+ 353us) server_negotiation.cc:407] Connection header received
0927 07:01:08.521901 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.521908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.521991 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.522055 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:08.522911 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.524043 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.525952 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.526718 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.526873 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.527034 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.527369 (+ 335us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.527524 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
I20260927 07:01:08.527849 21922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.520745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:35815 (local address 127.13.121.193:47521)
0927 07:01:08.521472 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:08.521494 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:08.521730 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:08.522263 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:08.522267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:08.522319 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:08.522724 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.522737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.524187 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.524193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.525782 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.525795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.526927 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.526934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.527204 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.527240 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.527339 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.527493 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":1379,"threads_started":1}
I20260927 07:01:08.528543 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28b27ddc4724bf5804e1f392e011c9f" candidate_uuid: "0e06e761b24e4c679f7e2cf307d5e763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9" is_pre_election: true
I20260927 07:01:08.528836 21807 raft_consensus.cc:2454] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e06e761b24e4c679f7e2cf307d5e763 in term 0.
I20260927 07:01:08.529136 21921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.520786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42687 (local address 127.13.121.193:47623)
0927 07:01:08.522422 (+ 1636us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:08.522442 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:08.522593 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:08.522970 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:08.522975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:08.522999 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:08.523342 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.523350 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.524534 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.524540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.526604 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.526621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.528345 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.528350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.528532 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.528564 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.528660 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.528796 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1479,"thread_start_us":1033,"threads_started":1}
I20260927 07:01:08.529415 21459 leader_election.cc:302] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e06e761b24e4c679f7e2cf307d5e763, 260e9d6fc9284eb69821e8194f29dbb9; no voters:
I20260927 07:01:08.529724 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.521619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:47623 (local address 127.13.121.194:42687)
0927 07:01:08.521810 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.521816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.522561 (+ 745us) server_negotiation.cc:407] Connection header received
0927 07:01:08.522736 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.522742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.522841 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.522947 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:08.523500 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.524378 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.526779 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.527995 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.528141 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.528210 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.528800 (+ 590us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.529016 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 07:01:08.529783 21891 raft_consensus.cc:2781] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:01:08.529901 21891 raft_consensus.cc:482] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:01:08.529983 21891 raft_consensus.cc:3037] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.530644 21668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28b27ddc4724bf5804e1f392e011c9f" candidate_uuid: "0e06e761b24e4c679f7e2cf307d5e763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a028898f5c4640c3afcdb1b13c45c410" is_pre_election: true
I20260927 07:01:08.531188 21668 raft_consensus.cc:2454] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e06e761b24e4c679f7e2cf307d5e763 in term 0.
I20260927 07:01:08.533046 21891 raft_consensus.cc:504] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.533689 21891 leader_election.cc:288] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [CANDIDATE]: Term 1 election: Requested vote from peers a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194:42687), 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815)
I20260927 07:01:08.534241 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28b27ddc4724bf5804e1f392e011c9f" candidate_uuid: "0e06e761b24e4c679f7e2cf307d5e763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9"
I20260927 07:01:08.534269 21668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28b27ddc4724bf5804e1f392e011c9f" candidate_uuid: "0e06e761b24e4c679f7e2cf307d5e763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a028898f5c4640c3afcdb1b13c45c410"
I20260927 07:01:08.534482 21807 raft_consensus.cc:3037] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.534503 21668 raft_consensus.cc:3037] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.536550 21807 raft_consensus.cc:2454] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e06e761b24e4c679f7e2cf307d5e763 in term 1.
I20260927 07:01:08.537055 21459 leader_election.cc:302] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e06e761b24e4c679f7e2cf307d5e763, 260e9d6fc9284eb69821e8194f29dbb9; no voters:
I20260927 07:01:08.537348 21891 raft_consensus.cc:2781] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:08.537802 21891 raft_consensus.cc:686] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 LEADER]: Becoming Leader. State: Replica: 0e06e761b24e4c679f7e2cf307d5e763, State: Running, Role: LEADER
I20260927 07:01:08.537911 21668 raft_consensus.cc:2454] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e06e761b24e4c679f7e2cf307d5e763 in term 1.
I20260927 07:01:08.538247 21891 consensus_queue.cc:237] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.540545 21923 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: TxnStatusTablet state changed. Reason: New leader 0e06e761b24e4c679f7e2cf307d5e763. Latest consensus state: current_term: 1 leader_uuid: "0e06e761b24e4c679f7e2cf307d5e763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.540966 21923 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:01:08.541656 21925 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 07:01:08.541844 21925 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 07:01:08.542184 21371 catalog_manager.cc:5495] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "0e06e761b24e4c679f7e2cf307d5e763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } health_report { overall_health: UNKNOWN } } }
I20260927 07:01:08.564262 21917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.558588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43443 (local address 127.0.0.1:35098)
0927 07:01:08.558818 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:08.558838 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:08.558997 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:08.559516 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:08.559522 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:08.559534 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:01:08.559696 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.559705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.560738 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.560743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.561718 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.561731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.562427 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.562434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.562601 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.562662 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:01:08.563734 (+ 1072us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:01:08.563752 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.563822 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.563981 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 07:01:08.564303 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.558745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35098 (local address 127.13.121.193:43443)
0927 07:01:08.558907 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.558912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.558932 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:01:08.559219 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.559227 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.559343 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.559481 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:08.559818 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.560605 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.561856 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.562274 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.562385 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.562864 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:08.563549 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:08.563687 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.563862 (+ 175us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.563959 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 07:01:08.568491 21807 raft_consensus.cc:1261] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Refusing update from remote peer 0e06e761b24e4c679f7e2cf307d5e763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:08.568944 21668 raft_consensus.cc:1261] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Refusing update from remote peer 0e06e761b24e4c679f7e2cf307d5e763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:08.569116 21891 consensus_queue.cc:1035] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:08.570955 21891 consensus_queue.cc:1035] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:08.572144 21888 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: TxnStatusTablet state changed. Reason: New leader 0e06e761b24e4c679f7e2cf307d5e763. Latest consensus state: current_term: 1 leader_uuid: "0e06e761b24e4c679f7e2cf307d5e763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.572352 21888 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:01:08.575451 21923 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e06e761b24e4c679f7e2cf307d5e763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.575690 21923 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:01:08.580184 21888 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e06e761b24e4c679f7e2cf307d5e763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.580442 21888 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P 260e9d6fc9284eb69821e8194f29dbb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:01:08.596531 21902 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: TxnStatusTablet state changed. Reason: New leader 0e06e761b24e4c679f7e2cf307d5e763. Latest consensus state: current_term: 1 leader_uuid: "0e06e761b24e4c679f7e2cf307d5e763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.596863 21902 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:01:08.598248 21891 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e06e761b24e4c679f7e2cf307d5e763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.598604 21891 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P 0e06e761b24e4c679f7e2cf307d5e763: This TxnStatusTablet replica's current role is: LEADER
I20260927 07:01:08.601368 21902 tablet_replica.cc:439] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e06e761b24e4c679f7e2cf307d5e763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } } }
I20260927 07:01:08.601745 21902 tablet_replica.cc:442] T f28b27ddc4724bf5804e1f392e011c9f P a028898f5c4640c3afcdb1b13c45c410: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 07:01:08.613453 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.605912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:34621 (local address 127.13.121.193:43443)
0927 07:01:08.606148 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.606154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.606776 (+ 622us) server_negotiation.cc:407] Connection header received
0927 07:01:08.606972 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.606978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.607070 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.607208 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:08.607781 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.608957 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.611040 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.611816 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.611953 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.612562 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:08.612720 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:08.612826 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.613014 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.613112 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:01:08.613557 21932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.605809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43443 (local address 127.13.121.194:34621)
0927 07:01:08.606646 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:08.606669 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:08.606830 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:08.607245 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:08.607250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:08.607267 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:01:08.607601 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.607614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.609107 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.609113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.610856 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.610873 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.612018 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.612025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.612244 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.612326 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:01:08.612844 (+ 518us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:01:08.612857 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.612949 (+ 92us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.613117 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":484,"threads_started":1}
I20260927 07:01:08.625785 21807 raft_consensus.cc:1261] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Refusing update from remote peer a028898f5c4640c3afcdb1b13c45c410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:08.625806 21527 raft_consensus.cc:1261] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Refusing update from remote peer a028898f5c4640c3afcdb1b13c45c410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:08.626528 21886 consensus_queue.cc:1035] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:08.626833 21902 consensus_queue.cc:1035] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:08.674883 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.667452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:49423 (local address 127.13.121.194:42687)
0927 07:01:08.667662 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.667668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.668294 (+ 626us) server_negotiation.cc:407] Connection header received
0927 07:01:08.668450 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.668456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.668545 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.668640 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:08.669173 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.670312 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.672564 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.673205 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.673337 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.673852 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:08.673965 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:08.674043 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.674521 (+ 478us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.674609 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 07:01:08.675580 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.667483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42687 (local address 127.13.121.193:49423)
0927 07:01:08.668186 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:08.668202 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:08.668329 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:08.668670 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:08.668674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:08.668684 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:01:08.669006 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.669018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.670467 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.670472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.672369 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:08.672384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.673413 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.673420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:08.673575 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.673665 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:01:08.674371 (+ 706us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:01:08.674379 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:01:08.675086 (+ 707us) client_negotiation.cc:240] Negotiation successful
0927 07:01:08.675274 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":326,"threads_started":1}
I20260927 07:01:08.716189 21371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:08.716446 21371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:08.719620 21371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb: Sending DeleteTablet for 3 replicas of tablet d42e20d9d7db47cd866eb3becb70f36e
I20260927 07:01:08.720520 21648 tablet_service.cc:1514] Processing DeleteTablet for tablet d42e20d9d7db47cd866eb3becb70f36e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:54110
I20260927 07:01:08.720615 21508 tablet_service.cc:1514] Processing DeleteTablet for tablet d42e20d9d7db47cd866eb3becb70f36e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35088
I20260927 07:01:08.720804 21905 tablet_replica.cc:330] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: stopping tablet replica
I20260927 07:01:08.720901 21906 tablet_replica.cc:330] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: stopping tablet replica
I20260927 07:01:08.720958 21787 tablet_service.cc:1514] Processing DeleteTablet for tablet d42e20d9d7db47cd866eb3becb70f36e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:59582
I20260927 07:01:08.721004 21905 raft_consensus.cc:2229] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:01:08.721108 21906 raft_consensus.cc:2229] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:08.721230 21907 tablet_replica.cc:330] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: stopping tablet replica
I20260927 07:01:08.721315 21906 raft_consensus.cc:2258] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:08.721319 21905 raft_consensus.cc:2258] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:08.721403 21907 raft_consensus.cc:2229] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:08.721593 21907 raft_consensus.cc:2258] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:08.722523 21906 ts_tablet_manager.cc:1905] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:08.722509 21907 ts_tablet_manager.cc:1905] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:08.722592 21905 ts_tablet_manager.cc:1905] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:08.729523 21905 ts_tablet_manager.cc:1918] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:01:08.729573 21906 ts_tablet_manager.cc:1918] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:01:08.729689 21905 log.cc:1196] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/d42e20d9d7db47cd866eb3becb70f36e
I20260927 07:01:08.729728 21906 log.cc:1196] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/d42e20d9d7db47cd866eb3becb70f36e
I20260927 07:01:08.730067 21905 ts_tablet_manager.cc:1939] T d42e20d9d7db47cd866eb3becb70f36e P a028898f5c4640c3afcdb1b13c45c410: Deleting consensus metadata
I20260927 07:01:08.730176 21906 ts_tablet_manager.cc:1939] T d42e20d9d7db47cd866eb3becb70f36e P 0e06e761b24e4c679f7e2cf307d5e763: Deleting consensus metadata
I20260927 07:01:08.731166 21358 catalog_manager.cc:4841] TS a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194:42687): tablet d42e20d9d7db47cd866eb3becb70f36e (table test-table [id=0d4c20dd8db84f97a674e08f97d6a90b]) successfully deleted
I20260927 07:01:08.731258 21356 catalog_manager.cc:4841] TS 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443): tablet d42e20d9d7db47cd866eb3becb70f36e (table test-table [id=0d4c20dd8db84f97a674e08f97d6a90b]) successfully deleted
I20260927 07:01:08.732213 21907 ts_tablet_manager.cc:1918] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:01:08.732362 21907 log.cc:1196] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/d42e20d9d7db47cd866eb3becb70f36e
I20260927 07:01:08.732753 21907 ts_tablet_manager.cc:1939] T d42e20d9d7db47cd866eb3becb70f36e P 260e9d6fc9284eb69821e8194f29dbb9: Deleting consensus metadata
I20260927 07:01:08.733788 21356 catalog_manager.cc:4841] TS 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815): tablet d42e20d9d7db47cd866eb3becb70f36e (table test-table [id=0d4c20dd8db84f97a674e08f97d6a90b]) successfully deleted
Sep 27 07:01:08 dist-test-slave-p6hz krb5kdc[21326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 07:01:08 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:08 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 07:01:08.782006 21606 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:08.771516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53166 (local address 127.13.121.254:36709)
0927 07:01:08.771706 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.771713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.772120 (+ 407us) server_negotiation.cc:407] Connection header received
0927 07:01:08.772316 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.772322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.772431 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.772548 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:08.773755 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.774902 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.776165 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.776815 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.776949 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.781740 (+ 4791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53166: BlockingRecv error: recv got EOF from 127.0.0.1:53166 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 27 07:01:08 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM
I20260927 07:01:08.802386 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.785330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53180 (local address 127.13.121.254:36709)
0927 07:01:08.785528 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.785535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.785799 (+ 264us) server_negotiation.cc:407] Connection header received
0927 07:01:08.785938 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.785944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.786028 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.786140 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:08.787279 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.788444 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.790217 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.790902 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.791039 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.795808 (+ 4769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:08.795816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:08.795821 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:08.795870 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:08.799212 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:08.800188 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:08.800193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:08.800197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:08.800370 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:08.800724 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:08.800729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:08.800732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:08.801155 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:08.801386 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.801798 (+ 412us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.802030 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:01:08.806608 21371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:01:08.807331 21371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:01:08.815237 21648 tablet_service.cc:1467] Processing CreateTablet for tablet 27e6437ce4b84b7aa939ea45513ef843 (DEFAULT_TABLE table=test-table [id=8f51f5b1a2b941f1b9561645b77e8759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.815397 21508 tablet_service.cc:1467] Processing CreateTablet for tablet 27e6437ce4b84b7aa939ea45513ef843 (DEFAULT_TABLE table=test-table [id=8f51f5b1a2b941f1b9561645b77e8759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.815691 21648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27e6437ce4b84b7aa939ea45513ef843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.815860 21508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27e6437ce4b84b7aa939ea45513ef843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.823340 21787 tablet_service.cc:1467] Processing CreateTablet for tablet 27e6437ce4b84b7aa939ea45513ef843 (DEFAULT_TABLE table=test-table [id=8f51f5b1a2b941f1b9561645b77e8759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.823793 21787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27e6437ce4b84b7aa939ea45513ef843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.824954 21881 tablet_bootstrap.cc:492] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: Bootstrap starting.
I20260927 07:01:08.826541 21881 tablet_bootstrap.cc:654] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.831698 21953 tablet_bootstrap.cc:492] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: Bootstrap starting.
I20260927 07:01:08.832046 21881 tablet_bootstrap.cc:492] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: No bootstrap required, opened a new log
I20260927 07:01:08.832239 21881 ts_tablet_manager.cc:1397] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260927 07:01:08.833137 21881 raft_consensus.cc:348] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.833343 21953 tablet_bootstrap.cc:654] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.833428 21881 raft_consensus.cc:374] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.833513 21881 raft_consensus.cc:729] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a028898f5c4640c3afcdb1b13c45c410, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.833799 21881 consensus_queue.cc:260] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.835439 21951 tablet_bootstrap.cc:492] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: Bootstrap starting.
I20260927 07:01:08.836580 21881 ts_tablet_manager.cc:1428] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260927 07:01:08.837091 21951 tablet_bootstrap.cc:654] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.839808 21951 tablet_bootstrap.cc:492] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: No bootstrap required, opened a new log
I20260927 07:01:08.839995 21951 ts_tablet_manager.cc:1397] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 07:01:08.840273 21953 tablet_bootstrap.cc:492] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: No bootstrap required, opened a new log
I20260927 07:01:08.840423 21953 ts_tablet_manager.cc:1397] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260927 07:01:08.840896 21951 raft_consensus.cc:348] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.841193 21951 raft_consensus.cc:374] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.841153 21953 raft_consensus.cc:348] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.841317 21951 raft_consensus.cc:729] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e06e761b24e4c679f7e2cf307d5e763, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.841384 21953 raft_consensus.cc:374] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.841464 21953 raft_consensus.cc:729] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260e9d6fc9284eb69821e8194f29dbb9, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.841596 21951 consensus_queue.cc:260] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.841702 21953 consensus_queue.cc:260] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.842077 21886 raft_consensus.cc:482] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:01:08.842216 21953 ts_tablet_manager.cc:1428] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260927 07:01:08.842229 21886 raft_consensus.cc:504] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.842825 21886 leader_election.cc:288] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443), 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815)
I20260927 07:01:08.843649 21528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e6437ce4b84b7aa939ea45513ef843" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e06e761b24e4c679f7e2cf307d5e763" is_pre_election: true
I20260927 07:01:08.843866 21528 raft_consensus.cc:2454] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 0.
I20260927 07:01:08.844205 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e6437ce4b84b7aa939ea45513ef843" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9" is_pre_election: true
I20260927 07:01:08.844410 21807 raft_consensus.cc:2454] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 0.
I20260927 07:01:08.844985 21599 leader_election.cc:302] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e06e761b24e4c679f7e2cf307d5e763, a028898f5c4640c3afcdb1b13c45c410; no voters:
I20260927 07:01:08.845431 21951 ts_tablet_manager.cc:1428] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260927 07:01:08.845667 21886 raft_consensus.cc:2781] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:01:08.845778 21886 raft_consensus.cc:482] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:01:08.845860 21886 raft_consensus.cc:3037] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.848604 21886 raft_consensus.cc:504] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.849092 21886 leader_election.cc:288] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 election: Requested vote from peers 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443), 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815)
I20260927 07:01:08.849627 21528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e6437ce4b84b7aa939ea45513ef843" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e06e761b24e4c679f7e2cf307d5e763"
I20260927 07:01:08.849848 21528 raft_consensus.cc:3037] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.851411 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e6437ce4b84b7aa939ea45513ef843" candidate_uuid: "a028898f5c4640c3afcdb1b13c45c410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9"
I20260927 07:01:08.851635 21807 raft_consensus.cc:3037] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.852463 21528 raft_consensus.cc:2454] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 1.
I20260927 07:01:08.852921 21599 leader_election.cc:302] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e06e761b24e4c679f7e2cf307d5e763, a028898f5c4640c3afcdb1b13c45c410; no voters:
I20260927 07:01:08.853201 21886 raft_consensus.cc:2781] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:08.853317 21886 raft_consensus.cc:686] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 1 LEADER]: Becoming Leader. State: Replica: a028898f5c4640c3afcdb1b13c45c410, State: Running, Role: LEADER
I20260927 07:01:08.853626 21886 consensus_queue.cc:237] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.855513 21807 raft_consensus.cc:2454] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a028898f5c4640c3afcdb1b13c45c410 in term 1.
I20260927 07:01:08.856361 21371 catalog_manager.cc:5495] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 reported cstate change: term changed from 0 to 1, leader changed from <none> to a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194). New cstate: current_term: 1 leader_uuid: "a028898f5c4640c3afcdb1b13c45c410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } health_report { overall_health: UNKNOWN } } }
I20260927 07:01:08.876127 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:08.870283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54126 (local address 127.13.121.194:42687)
0927 07:01:08.870452 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:08.870457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:08.870716 (+ 259us) server_negotiation.cc:407] Connection header received
0927 07:01:08.870902 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:08.870905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:08.870969 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:08.871041 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:08.871507 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.872396 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.874368 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:08.874785 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:08.874894 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:08.875424 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:08.875559 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:08.875630 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:08.875789 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 07:01:08.875859 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:01:08.878638 21528 raft_consensus.cc:1261] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Refusing update from remote peer a028898f5c4640c3afcdb1b13c45c410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:08.878810 21807 raft_consensus.cc:1261] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Refusing update from remote peer a028898f5c4640c3afcdb1b13c45c410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:08.879302 21892 consensus_queue.cc:1035] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:08.879920 21886 consensus_queue.cc:1035] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:08.897810 21371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:08.898073 21371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:08.901408 21371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb: Sending DeleteTablet for 3 replicas of tablet 27e6437ce4b84b7aa939ea45513ef843
I20260927 07:01:08.902292 21787 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e6437ce4b84b7aa939ea45513ef843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:59582
I20260927 07:01:08.902637 21508 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e6437ce4b84b7aa939ea45513ef843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35088
I20260927 07:01:08.902781 21907 tablet_replica.cc:330] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: stopping tablet replica
I20260927 07:01:08.902976 21907 raft_consensus.cc:2229] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:08.903012 21648 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e6437ce4b84b7aa939ea45513ef843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:08 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:54110
I20260927 07:01:08.903347 21906 tablet_replica.cc:330] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: stopping tablet replica
I20260927 07:01:08.903328 21907 raft_consensus.cc:2258] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:08.903548 21906 raft_consensus.cc:2229] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:08.903805 21906 raft_consensus.cc:2258] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:08.904084 21905 tablet_replica.cc:330] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: stopping tablet replica
I20260927 07:01:08.904251 21905 raft_consensus.cc:2229] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:01:08.904424 21907 ts_tablet_manager.cc:1905] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:08.904265 21371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 07:01:08.904614 21905 raft_consensus.cc:2258] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 07:01:08.905171 21371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:01:08.905925 21905 ts_tablet_manager.cc:1905] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:08.910861 21907 ts_tablet_manager.cc:1918] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:08.911020 21907 log.cc:1196] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/27e6437ce4b84b7aa939ea45513ef843
I20260927 07:01:08.911525 21907 ts_tablet_manager.cc:1939] T 27e6437ce4b84b7aa939ea45513ef843 P 260e9d6fc9284eb69821e8194f29dbb9: Deleting consensus metadata
I20260927 07:01:08.912364 21905 ts_tablet_manager.cc:1918] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:08.912490 21905 log.cc:1196] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/27e6437ce4b84b7aa939ea45513ef843
I20260927 07:01:08.912644 21356 catalog_manager.cc:4841] TS 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815): tablet 27e6437ce4b84b7aa939ea45513ef843 (table test-table [id=8f51f5b1a2b941f1b9561645b77e8759]) successfully deleted
I20260927 07:01:08.912854 21905 ts_tablet_manager.cc:1939] T 27e6437ce4b84b7aa939ea45513ef843 P a028898f5c4640c3afcdb1b13c45c410: Deleting consensus metadata
I20260927 07:01:08.913642 21787 tablet_service.cc:1467] Processing CreateTablet for tablet 5a35d58bb7a043f48b293a84e7a78fd3 (DEFAULT_TABLE table=test-table [id=0d6787e9151e46d9aae5b1dd60737f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.913727 21648 tablet_service.cc:1467] Processing CreateTablet for tablet 5a35d58bb7a043f48b293a84e7a78fd3 (DEFAULT_TABLE table=test-table [id=0d6787e9151e46d9aae5b1dd60737f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.914052 21787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a35d58bb7a043f48b293a84e7a78fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.914052 21648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a35d58bb7a043f48b293a84e7a78fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.919582 21508 tablet_service.cc:1467] Processing CreateTablet for tablet 5a35d58bb7a043f48b293a84e7a78fd3 (DEFAULT_TABLE table=test-table [id=0d6787e9151e46d9aae5b1dd60737f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:08.920107 21508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a35d58bb7a043f48b293a84e7a78fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:08.920415 21953 tablet_bootstrap.cc:492] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: Bootstrap starting.
I20260927 07:01:08.921959 21953 tablet_bootstrap.cc:654] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.923857 21906 ts_tablet_manager.cc:1905] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:08.924708 21953 tablet_bootstrap.cc:492] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: No bootstrap required, opened a new log
I20260927 07:01:08.924880 21953 ts_tablet_manager.cc:1397] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 07:01:08.925660 21953 raft_consensus.cc:348] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.925918 21953 raft_consensus.cc:374] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.926004 21953 raft_consensus.cc:729] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260e9d6fc9284eb69821e8194f29dbb9, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.926247 21953 consensus_queue.cc:260] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.926824 21953 ts_tablet_manager.cc:1428] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:01:08.931042 21951 tablet_bootstrap.cc:492] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: Bootstrap starting.
I20260927 07:01:08.933044 21951 tablet_bootstrap.cc:654] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.934732 21906 ts_tablet_manager.cc:1918] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:08.934918 21906 log.cc:1196] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/27e6437ce4b84b7aa939ea45513ef843
I20260927 07:01:08.935525 21881 tablet_bootstrap.cc:492] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: Bootstrap starting.
I20260927 07:01:08.937354 21881 tablet_bootstrap.cc:654] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:08.938984 21358 catalog_manager.cc:4841] TS a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194:42687): tablet 27e6437ce4b84b7aa939ea45513ef843 (table test-table [id=8f51f5b1a2b941f1b9561645b77e8759]) successfully deleted
I20260927 07:01:08.942360 21906 ts_tablet_manager.cc:1939] T 27e6437ce4b84b7aa939ea45513ef843 P 0e06e761b24e4c679f7e2cf307d5e763: Deleting consensus metadata
I20260927 07:01:08.944407 21356 catalog_manager.cc:4841] TS 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443): tablet 27e6437ce4b84b7aa939ea45513ef843 (table test-table [id=8f51f5b1a2b941f1b9561645b77e8759]) successfully deleted
I20260927 07:01:08.949999 21881 tablet_bootstrap.cc:492] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: No bootstrap required, opened a new log
I20260927 07:01:08.950345 21881 ts_tablet_manager.cc:1397] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.007s
I20260927 07:01:08.951540 21881 raft_consensus.cc:348] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.952036 21881 raft_consensus.cc:374] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.952173 21881 raft_consensus.cc:729] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a028898f5c4640c3afcdb1b13c45c410, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.952589 21881 consensus_queue.cc:260] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.953305 21881 ts_tablet_manager.cc:1428] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 07:01:08.955305 21951 tablet_bootstrap.cc:492] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: No bootstrap required, opened a new log
I20260927 07:01:08.955498 21951 ts_tablet_manager.cc:1397] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: Time spent bootstrapping tablet: real 0.025s user 0.015s sys 0.008s
I20260927 07:01:08.956357 21951 raft_consensus.cc:348] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.956633 21951 raft_consensus.cc:374] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:08.956717 21951 raft_consensus.cc:729] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e06e761b24e4c679f7e2cf307d5e763, State: Initialized, Role: FOLLOWER
I20260927 07:01:08.956996 21951 consensus_queue.cc:260] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.957609 21951 ts_tablet_manager.cc:1428] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 07:01:08.981916 21923 raft_consensus.cc:482] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:01:08.982159 21923 raft_consensus.cc:504] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.982976 21923 leader_election.cc:288] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194:42687), 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815)
I20260927 07:01:08.983597 21668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a35d58bb7a043f48b293a84e7a78fd3" candidate_uuid: "0e06e761b24e4c679f7e2cf307d5e763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a028898f5c4640c3afcdb1b13c45c410" is_pre_election: true
I20260927 07:01:08.983601 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a35d58bb7a043f48b293a84e7a78fd3" candidate_uuid: "0e06e761b24e4c679f7e2cf307d5e763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9" is_pre_election: true
I20260927 07:01:08.983844 21807 raft_consensus.cc:2454] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e06e761b24e4c679f7e2cf307d5e763 in term 0.
I20260927 07:01:08.983865 21668 raft_consensus.cc:2454] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e06e761b24e4c679f7e2cf307d5e763 in term 0.
I20260927 07:01:08.984362 21459 leader_election.cc:302] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0e06e761b24e4c679f7e2cf307d5e763, 260e9d6fc9284eb69821e8194f29dbb9, a028898f5c4640c3afcdb1b13c45c410; no voters:
I20260927 07:01:08.984727 21923 raft_consensus.cc:2781] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:01:08.984866 21923 raft_consensus.cc:482] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:01:08.984989 21923 raft_consensus.cc:3037] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.987349 21923 raft_consensus.cc:504] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.987962 21923 leader_election.cc:288] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [CANDIDATE]: Term 1 election: Requested vote from peers a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194:42687), 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815)
I20260927 07:01:08.988509 21807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a35d58bb7a043f48b293a84e7a78fd3" candidate_uuid: "0e06e761b24e4c679f7e2cf307d5e763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e9d6fc9284eb69821e8194f29dbb9"
I20260927 07:01:08.988531 21668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a35d58bb7a043f48b293a84e7a78fd3" candidate_uuid: "0e06e761b24e4c679f7e2cf307d5e763" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a028898f5c4640c3afcdb1b13c45c410"
I20260927 07:01:08.988701 21807 raft_consensus.cc:3037] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.988740 21668 raft_consensus.cc:3037] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:08.990630 21807 raft_consensus.cc:2454] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e06e761b24e4c679f7e2cf307d5e763 in term 1.
I20260927 07:01:08.991058 21459 leader_election.cc:302] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e06e761b24e4c679f7e2cf307d5e763, 260e9d6fc9284eb69821e8194f29dbb9; no voters:
I20260927 07:01:08.991406 21923 raft_consensus.cc:2781] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:08.991638 21923 raft_consensus.cc:686] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 LEADER]: Becoming Leader. State: Replica: 0e06e761b24e4c679f7e2cf307d5e763, State: Running, Role: LEADER
I20260927 07:01:08.991722 21668 raft_consensus.cc:2454] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e06e761b24e4c679f7e2cf307d5e763 in term 1.
I20260927 07:01:08.991914 21923 consensus_queue.cc:237] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } }
I20260927 07:01:08.994850 21369 catalog_manager.cc:5495] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "0e06e761b24e4c679f7e2cf307d5e763" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e06e761b24e4c679f7e2cf307d5e763" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 } health_report { overall_health: UNKNOWN } } }
I20260927 07:01:09.018817 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:09.013196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35102 (local address 127.13.121.193:43443)
0927 07:01:09.013339 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:09.013344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:09.013409 (+ 65us) server_negotiation.cc:407] Connection header received
0927 07:01:09.013594 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:09.013597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:09.013660 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:09.013722 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:09.014239 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.015087 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.017035 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.017511 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.017622 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:09.018131 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:09.018262 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:09.018338 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:09.018489 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 07:01:09.018557 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 07:01:09.033342 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:09.027901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:34261 (local address 127.13.121.193:43443)
0927 07:01:09.028084 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:09.028090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:09.028129 (+ 39us) server_negotiation.cc:407] Connection header received
0927 07:01:09.028321 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:09.028325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:09.028381 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:09.028453 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:09.028817 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.029566 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.031480 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.032006 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.032113 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:09.032572 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:09.032723 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:09.032792 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:09.032981 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 07:01:09.033082 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":65}
I20260927 07:01:09.033417 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:09.027848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43443 (local address 127.13.121.193:34261)
0927 07:01:09.028037 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:09.028059 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:09.028200 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:09.028468 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:09.028472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:09.028489 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 07:01:09.028706 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:09.028714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.029699 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.029705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:09.031289 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:09.031307 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.032165 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.032172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:09.032344 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:09.032396 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 07:01:09.032801 (+ 405us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 07:01:09.032812 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 07:01:09.032904 (+ 92us) client_negotiation.cc:240] Negotiation successful
0927 07:01:09.033082 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":32}
I20260927 07:01:09.041596 21668 raft_consensus.cc:1261] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Refusing update from remote peer 0e06e761b24e4c679f7e2cf307d5e763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:09.041630 21807 raft_consensus.cc:1261] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Refusing update from remote peer 0e06e761b24e4c679f7e2cf307d5e763: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:09.042098 21891 consensus_queue.cc:1035] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a028898f5c4640c3afcdb1b13c45c410" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:09.042343 21923 consensus_queue.cc:1035] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [LEADER]: Connected to new peer: Peer: permanent_uuid: "260e9d6fc9284eb69821e8194f29dbb9" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:09.096778 21369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:09.097033 21369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:09.100327 21369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63d5b3b7179547d593194a0575c81efb: Sending DeleteTablet for 3 replicas of tablet 5a35d58bb7a043f48b293a84e7a78fd3
I20260927 07:01:09.101223 21648 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a35d58bb7a043f48b293a84e7a78fd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:09 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:54110
I20260927 07:01:09.101243 21787 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a35d58bb7a043f48b293a84e7a78fd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:09 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:59582
I20260927 07:01:09.101549 21905 tablet_replica.cc:330] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: stopping tablet replica
I20260927 07:01:09.101759 21905 raft_consensus.cc:2229] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:09.101965 21905 raft_consensus.cc:2258] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:09.102291 21907 tablet_replica.cc:330] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: stopping tablet replica
I20260927 07:01:09.102607 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 21442
I20260927 07:01:09.102617 21907 raft_consensus.cc:2229] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:09.102821 21907 raft_consensus.cc:2258] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:09.103185 21905 ts_tablet_manager.cc:1905] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:09.103986 21907 ts_tablet_manager.cc:1905] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:09.104732 21508 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a35d58bb7a043f48b293a84e7a78fd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:09 UTC) from {username='oryx', principal='oryx/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35088
I20260927 07:01:09.105047 21906 tablet_replica.cc:330] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: stopping tablet replica
I20260927 07:01:09.105257 21906 raft_consensus.cc:2229] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:01:09.105904 21906 raft_consensus.cc:2258] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:09.107677 21906 ts_tablet_manager.cc:1905] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:09.120116 21906 ts_tablet_manager.cc:1918] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:01:09.122408 21905 ts_tablet_manager.cc:1918] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:01:09.122901 21905 log.cc:1196] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/5a35d58bb7a043f48b293a84e7a78fd3
I20260927 07:01:09.124070 21907 ts_tablet_manager.cc:1918] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 07:01:09.124260 21907 log.cc:1196] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/5a35d58bb7a043f48b293a84e7a78fd3
I20260927 07:01:09.124500 21906 log.cc:1196] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/5a35d58bb7a043f48b293a84e7a78fd3
I20260927 07:01:09.124668 21905 ts_tablet_manager.cc:1939] T 5a35d58bb7a043f48b293a84e7a78fd3 P a028898f5c4640c3afcdb1b13c45c410: Deleting consensus metadata
I20260927 07:01:09.124679 21907 ts_tablet_manager.cc:1939] T 5a35d58bb7a043f48b293a84e7a78fd3 P 260e9d6fc9284eb69821e8194f29dbb9: Deleting consensus metadata
Sep 27 07:01:09 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.193@KRBTEST.COM
I20260927 07:01:09.125279 21906 ts_tablet_manager.cc:1939] T 5a35d58bb7a043f48b293a84e7a78fd3 P 0e06e761b24e4c679f7e2cf307d5e763: Deleting consensus metadata
I20260927 07:01:09.125799 21356 catalog_manager.cc:4841] TS 260e9d6fc9284eb69821e8194f29dbb9 (127.13.121.195:35815): tablet 5a35d58bb7a043f48b293a84e7a78fd3 (table test-table [id=0d6787e9151e46d9aae5b1dd60737f0d]) successfully deleted
I20260927 07:01:09.127305 21356 catalog_manager.cc:4841] TS 0e06e761b24e4c679f7e2cf307d5e763 (127.13.121.193:43443): tablet 5a35d58bb7a043f48b293a84e7a78fd3 (table test-table [id=0d6787e9151e46d9aae5b1dd60737f0d]) successfully deleted
I20260927 07:01:09.127554 21358 catalog_manager.cc:4841] TS a028898f5c4640c3afcdb1b13c45c410 (127.13.121.194:42687): tablet 5a35d58bb7a043f48b293a84e7a78fd3 (table test-table [id=0d6787e9151e46d9aae5b1dd60737f0d]) successfully deleted
I20260927 07:01:09.132522 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:09.107511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35114 (local address 127.13.121.193:43443)
0927 07:01:09.107678 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:09.107684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:09.114550 (+ 6866us) server_negotiation.cc:407] Connection header received
0927 07:01:09.115047 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:09.115053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:09.115166 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:09.115238 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:09.116952 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.118064 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.120005 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.120679 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.120781 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:09.126035 (+ 5254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:09.126058 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:09.126079 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:09.126129 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:09.129622 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:09.130374 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:09.130384 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:09.130387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:09.130551 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:09.130922 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:09.130927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:09.130931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:09.131380 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:09.131688 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:09.132006 (+ 318us) server_negotiation.cc:299] Negotiation successful
0927 07:01:09.132227 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 07:01:09.133821 21573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:09.263602 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 21442
I20260927 07:01:09.283005 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 21582
Sep 27 07:01:09 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.194@KRBTEST.COM
I20260927 07:01:09.298019 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:09.283762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54138 (local address 127.13.121.194:42687)
0927 07:01:09.283912 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:09.283916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:09.283950 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:01:09.284111 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:09.284115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:09.284185 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:09.284273 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:09.285393 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.286091 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.287511 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.287943 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.288046 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:09.292061 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:09.292078 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:09.292101 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:09.292141 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:09.295058 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:09.295940 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:09.295949 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:09.295952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:09.296146 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:09.296563 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:09.296567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:09.296569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:09.296941 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:09.297208 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:09.297508 (+ 300us) server_negotiation.cc:299] Negotiation successful
0927 07:01:09.297689 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:01:09.298826 21712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:09.421954 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 21582
I20260927 07:01:09.439846 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 21722
Sep 27 07:01:09 dist-test-slave-p6hz krb5kdc[21326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.121.195@KRBTEST.COM
I20260927 07:01:09.456416 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:09.440564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59592 (local address 127.13.121.195:35815)
0927 07:01:09.441040 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:09.441045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:09.441063 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:01:09.441114 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:09.441118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:09.441181 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:09.441287 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:09.442777 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.443724 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.444976 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:09.445427 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:09.445539 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:09.450307 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:09.450324 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:09.450342 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:09.450385 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:09.453465 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:09.454282 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:09.454291 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:09.454293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:09.454442 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:09.454821 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:09.454825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:09.454827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:09.455386 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:09.455625 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:09.455890 (+ 265us) server_negotiation.cc:299] Negotiation successful
0927 07:01:09.456082 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":267,"threads_started":1}
I20260927 07:01:09.457231 21852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:09.552152 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 21722
I20260927 07:01:09.566047 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 21339
I20260927 07:01:09.567276 21411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:09.678248 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 21339
2026-09-27T07:01:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5240 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:01:09 dist-test-slave-p6hz krb5kdc[21975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:01:09 dist-test-slave-p6hz krb5kdc[21975](info): set up 2 sockets
Sep 27 07:01:09 dist-test-slave-p6hz krb5kdc[21975](info): commencing operation
krb5kdc: starting...
W20260927 07:01:10.704201 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:01:10 dist-test-slave-p6hz krb5kdc[21975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:01:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:01:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:01:10.919168 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:36387
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:43249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:36387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:11.240355 21988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:11.240638 21988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:11.240770 21988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:11.248092 21988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:01:11.248209 21988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:11.248279 21988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:11.248359 21988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:01:11.248425 21988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:01:11.258870 21988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:43249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:36387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:36387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:11.260264 21988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:11.261633 21988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:11.273577 21996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:11.273600 21994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:11.273577 21993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:11.274008 21988 server_base.cc:1034] running on GCE node
I20260927 07:01:11.274986 21988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:11.276549 21988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:11.277779 21988 hybrid_clock.cc:648] HybridClock initialized: now 1790492471277725 us; error 38 us; skew 500 ppm
I20260927 07:01:11.278168 21988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-27T07:01:11Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1669 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:01:11 dist-test-slave-p6hz krb5kdc[22002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:01:11 dist-test-slave-p6hz krb5kdc[22002](info): set up 2 sockets
Sep 27 07:01:11 dist-test-slave-p6hz krb5kdc[22002](info): commencing operation
krb5kdc: starting...
W20260927 07:01:12.368049 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:01:12 dist-test-slave-p6hz krb5kdc[22002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:01:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:01:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:01:12.583490 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38401
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:38401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:12.904269 22015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:12.904568 22015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:12.904677 22015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:12.911926 22015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:01:12.912040 22015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:12.912086 22015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:12.912163 22015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:01:12.912225 22015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:01:12.922078 22015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:38401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:12.923472 22015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:12.924775 22015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:12.936295 22020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:12.936313 22023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:12.936309 22021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:12.938028 22015 server_base.cc:1034] running on GCE node
I20260927 07:01:12.938935 22015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:12.940521 22015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:12.941756 22015 hybrid_clock.cc:648] HybridClock initialized: now 1790492472941697 us; error 49 us; skew 500 ppm
I20260927 07:01:12.942106 22015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:12 dist-test-slave-p6hz krb5kdc[22002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492472, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:12.945180 22015 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:01:12.946657 22015 webserver.cc:460] Webserver started at http://127.13.121.254:42395/ using document root <none> and password file <none>
I20260927 07:01:12.947357 22015 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:12.947469 22015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:12.947722 22015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:12.950726 22015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "27735d5fb8804135b8a1411744d1254c"
format_stamp: "Formatted at 2026-09-27 07:01:12 on dist-test-slave-p6hz"
server_key: "d86415e1245baea714edafcfb63e8de3"
server_key_iv: "375acfb1feda7c0735e2ce7698bdab6d"
server_key_version: "encryptionkey@0"
I20260927 07:01:12.951555 22015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "27735d5fb8804135b8a1411744d1254c"
format_stamp: "Formatted at 2026-09-27 07:01:12 on dist-test-slave-p6hz"
server_key: "d86415e1245baea714edafcfb63e8de3"
server_key_iv: "375acfb1feda7c0735e2ce7698bdab6d"
server_key_version: "encryptionkey@0"
I20260927 07:01:12.957358 22015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:01:12.961553 22030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:12.963207 22015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:01:12.963423 22015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "27735d5fb8804135b8a1411744d1254c"
format_stamp: "Formatted at 2026-09-27 07:01:12 on dist-test-slave-p6hz"
server_key: "d86415e1245baea714edafcfb63e8de3"
server_key_iv: "375acfb1feda7c0735e2ce7698bdab6d"
server_key_version: "encryptionkey@0"
I20260927 07:01:12.963630 22015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:12.976773 22015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:12.981547 22015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:12.981863 22015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:12.999397 22015 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:38401
I20260927 07:01:12.999433 22081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:38401 every 8 connection(s)
I20260927 07:01:13.000765 22015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:01:13.004470 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 22015
I20260927 07:01:13.004684 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:01:13.004765 22082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:13.005455 13799 external_mini_cluster.cc:1428] Setting key f24e3fcb0e71848d3ec785e59c14a7c9
I20260927 07:01:13.018810 22082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: Bootstrap starting.
Sep 27 07:01:13 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:01:13.022562 22082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:13.023723 22082 log.cc:824] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:13.027038 22082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: No bootstrap required, opened a new log
I20260927 07:01:13.030066 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:13.007634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.13.121.254:38401)
0927 07:01:13.008411 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:13.008427 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:13.008474 (+ 47us) server_negotiation.cc:407] Connection header received
0927 07:01:13.010993 (+ 2519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:13.011022 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:13.011531 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:13.011873 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:13.013679 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:13.014989 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:13.016801 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:13.017500 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:13.017643 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:13.022223 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:13.022232 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:13.022247 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:13.022289 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:13.025744 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:13.026597 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:13.026604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:13.026610 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:13.026784 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:13.027248 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:13.027262 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:13.027265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:13.027843 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:13.028109 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:13.028563 (+ 454us) server_negotiation.cc:299] Negotiation successful
0927 07:01:13.028937 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":353,"threads_started":1}
I20260927 07:01:13.035063 22082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } }
I20260927 07:01:13.035485 22082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:13.035583 22082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27735d5fb8804135b8a1411744d1254c, State: Initialized, Role: FOLLOWER
I20260927 07:01:13.036121 22082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } }
I20260927 07:01:13.036322 22082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:01:13.036443 22082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:01:13.036589 22082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:13.038722 22082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } }
I20260927 07:01:13.039309 22082 leader_election.cc:302] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27735d5fb8804135b8a1411744d1254c; no voters:
I20260927 07:01:13.039860 22082 leader_election.cc:288] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:01:13.039984 22087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:13.040620 22087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 1 LEADER]: Becoming Leader. State: Replica: 27735d5fb8804135b8a1411744d1254c, State: Running, Role: LEADER
I20260927 07:01:13.041129 22087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } }
I20260927 07:01:13.041867 22082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:01:13.044114 22089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27735d5fb8804135b8a1411744d1254c. Latest consensus state: current_term: 1 leader_uuid: "27735d5fb8804135b8a1411744d1254c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } } }
I20260927 07:01:13.044175 22088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27735d5fb8804135b8a1411744d1254c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } } }
I20260927 07:01:13.044476 22089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:13.044569 22088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:13.048184 22092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:01:13.052758 22092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:01:13.061801 22092 catalog_manager.cc:1347] Generated new cluster ID: 11c9012b4ceb4f31a04465299f6cf71b
I20260927 07:01:13.061934 22092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:01:13.072050 22092 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:01:13.074049 22092 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:01:13.082255 22092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: Generated new TSK 0
I20260927 07:01:13.083410 22092 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:01:13.144518 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38401
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:13.471256 22108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:13.471549 22108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:13.471665 22108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:13.479260 22108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:13.479387 22108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:13.479539 22108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:01:13.489471 22108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:13.490721 22108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:13.492045 22108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:13.504419 22113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:13.504436 22114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:13.504545 22116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:13.505708 22108 server_base.cc:1034] running on GCE node
I20260927 07:01:13.506311 22108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:13.507336 22108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:13.508553 22108 hybrid_clock.cc:648] HybridClock initialized: now 1790492473508501 us; error 45 us; skew 500 ppm
I20260927 07:01:13.508868 22108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:13 dist-test-slave-p6hz krb5kdc[22002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492473, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:13.511979 22108 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:01:13.513550 22108 webserver.cc:460] Webserver started at http://127.13.121.193:43045/ using document root <none> and password file <none>
I20260927 07:01:13.514251 22108 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:13.514371 22108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:13.514671 22108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:13.517678 22108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "cd0129f5d38b43e58a7eee84085cf65d"
format_stamp: "Formatted at 2026-09-27 07:01:13 on dist-test-slave-p6hz"
server_key: "731d5bd945020b7f0a4120a446dd6eee"
server_key_iv: "58b8dcdc7c4c069324281b0c30ae9a5f"
server_key_version: "encryptionkey@0"
I20260927 07:01:13.518460 22108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "cd0129f5d38b43e58a7eee84085cf65d"
format_stamp: "Formatted at 2026-09-27 07:01:13 on dist-test-slave-p6hz"
server_key: "731d5bd945020b7f0a4120a446dd6eee"
server_key_iv: "58b8dcdc7c4c069324281b0c30ae9a5f"
server_key_version: "encryptionkey@0"
I20260927 07:01:13.523819 22108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 07:01:13.527436 22123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:13.528887 22108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:01:13.529084 22108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "cd0129f5d38b43e58a7eee84085cf65d"
format_stamp: "Formatted at 2026-09-27 07:01:13 on dist-test-slave-p6hz"
server_key: "731d5bd945020b7f0a4120a446dd6eee"
server_key_iv: "58b8dcdc7c4c069324281b0c30ae9a5f"
server_key_version: "encryptionkey@0"
I20260927 07:01:13.529282 22108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:13.555473 22108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:13.559567 22108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:13.559803 22108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:13.560702 22108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:13.562016 22108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:13.562094 22108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:13.562175 22108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:13.562216 22108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:13.590441 22108 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:42621
I20260927 07:01:13.590478 22235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:42621 every 8 connection(s)
I20260927 07:01:13.591738 22108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:01:13.595757 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 22108
I20260927 07:01:13.595968 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:01:13.596534 13799 external_mini_cluster.cc:1428] Setting key 593771f36f282155206b0a8e6cf744c4
Sep 27 07:01:13 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492473, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:01:13.611723 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:13.594357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:34263 (local address 127.13.121.254:38401)
0927 07:01:13.594795 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:13.594799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:13.595670 (+ 871us) server_negotiation.cc:407] Connection header received
0927 07:01:13.597662 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:13.597668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:13.597757 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:13.597849 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:13.599471 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:13.600179 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:13.601734 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:13.602142 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:13.602256 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:13.606090 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:13.606096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:13.606099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:13.606134 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:13.608682 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:13.609421 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:13.609425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:13.609427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:13.609553 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:13.609959 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:13.609962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:13.609964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:13.610233 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:13.610361 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:13.611107 (+ 746us) server_negotiation.cc:299] Negotiation successful
0927 07:01:13.611365 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":224,"threads_started":1}
I20260927 07:01:13.612776 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:13.594783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:34263)
0927 07:01:13.595507 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:13.595550 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:13.597372 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:13.598120 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:13.598133 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:13.598667 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:13.599295 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:13.599317 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:13.600326 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:13.600329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:13.601591 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:13.601604 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:13.602380 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:13.602386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:13.602637 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:13.603342 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:13.603365 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:13.605846 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:13.608833 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:13.608843 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:13.608863 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:13.609289 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:13.609656 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:13.609661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:13.609665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:13.609848 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:13.610362 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:13.610370 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:13.610728 (+ 358us) client_negotiation.cc:769] Sending connection context
0927 07:01:13.610968 (+ 240us) client_negotiation.cc:240] Negotiation successful
0927 07:01:13.611555 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":306,"threads_started":1}
I20260927 07:01:13.614346 22236 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38401
I20260927 07:01:13.614634 22236 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:01:13.615389 22236 heartbeater.cc:511] Master 127.13.121.254:38401 requested a full tablet report, sending...
I20260927 07:01:13.617512 22047 ts_manager.cc:194] Registered new tserver with Master: cd0129f5d38b43e58a7eee84085cf65d (127.13.121.193:42621)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
I20260927 07:01:13.619071 22047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:34263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:01:13.645931 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38401
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:13.970598 22242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:13.970893 22242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:13.971020 22242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:13.978371 22242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:13.978492 22242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:13.978637 22242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:01:13.988792 22242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:13.990129 22242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:13.991482 22242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:14.005285 22247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:14.006024 22248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:14.006374 22250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:14.007542 22242 server_base.cc:1034] running on GCE node
I20260927 07:01:14.008213 22242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:14.009254 22242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:14.010522 22242 hybrid_clock.cc:648] HybridClock initialized: now 1790492474010440 us; error 54 us; skew 500 ppm
I20260927 07:01:14.010857 22242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:14 dist-test-slave-p6hz krb5kdc[22002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492474, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:14.014464 22242 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:01:14.016278 22242 webserver.cc:460] Webserver started at http://127.13.121.194:37317/ using document root <none> and password file <none>
I20260927 07:01:14.017171 22242 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:14.017303 22242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:14.017629 22242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:14.020917 22242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "8859519f30c94e80b371d39982dacd82"
format_stamp: "Formatted at 2026-09-27 07:01:14 on dist-test-slave-p6hz"
server_key: "5df5cc622eb0960998bdb6b60953c1d7"
server_key_iv: "bc46f249120071f546d1c32943b586ef"
server_key_version: "encryptionkey@0"
I20260927 07:01:14.021657 22242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "8859519f30c94e80b371d39982dacd82"
format_stamp: "Formatted at 2026-09-27 07:01:14 on dist-test-slave-p6hz"
server_key: "5df5cc622eb0960998bdb6b60953c1d7"
server_key_iv: "bc46f249120071f546d1c32943b586ef"
server_key_version: "encryptionkey@0"
I20260927 07:01:14.027281 22242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260927 07:01:14.031044 22257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:14.032857 22242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:01:14.033265 22242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "8859519f30c94e80b371d39982dacd82"
format_stamp: "Formatted at 2026-09-27 07:01:14 on dist-test-slave-p6hz"
server_key: "5df5cc622eb0960998bdb6b60953c1d7"
server_key_iv: "bc46f249120071f546d1c32943b586ef"
server_key_version: "encryptionkey@0"
I20260927 07:01:14.033623 22242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:14.055083 22242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:14.059857 22242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:14.060187 22242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:14.061223 22242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:14.062630 22242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:14.062706 22242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:14.062817 22242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:14.062878 22242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:14.096114 22242 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:42779
I20260927 07:01:14.096160 22369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:42779 every 8 connection(s)
I20260927 07:01:14.097791 22242 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:01:14.098287 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 22242
I20260927 07:01:14.098469 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:01:14.099014 13799 external_mini_cluster.cc:1428] Setting key 77dfe648049abc23b2979c9c2379ebfd
Sep 27 07:01:14 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492474, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:01:14.122776 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:14.102727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:55841 (local address 127.13.121.254:38401)
0927 07:01:14.103181 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:14.103188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:14.104830 (+ 1642us) server_negotiation.cc:407] Connection header received
0927 07:01:14.106859 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:14.106864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:14.106963 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:14.107067 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:14.108737 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.109534 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.111910 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.112555 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.112669 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:14.116828 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:14.116834 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:14.116837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:14.116875 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:14.119569 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:14.120395 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:14.120399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:14.120402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:14.120535 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:14.120982 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:14.120986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:14.120988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:14.121297 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:14.121429 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:14.122194 (+ 765us) server_negotiation.cc:299] Negotiation successful
0927 07:01:14.122435 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249}
I20260927 07:01:14.123942 22372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:14.103321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:55841)
0927 07:01:14.104471 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:14.104550 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:14.106559 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:14.107337 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:14.107347 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:14.107866 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:14.108534 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:14.108562 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.109694 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.109700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:14.111710 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:14.111728 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.112799 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.112806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:14.113094 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:14.113917 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:14.113945 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:14.116541 (+ 2596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:14.119759 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:14.119770 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:14.119787 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:14.120256 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:14.120647 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:14.120652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:14.120656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:14.120859 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:14.121424 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:14.121433 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:14.121818 (+ 385us) client_negotiation.cc:769] Sending connection context
0927 07:01:14.122068 (+ 250us) client_negotiation.cc:240] Negotiation successful
0927 07:01:14.122688 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":408,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
I20260927 07:01:14.125617 22370 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38401
I20260927 07:01:14.125985 22370 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:14.126859 22370 heartbeater.cc:511] Master 127.13.121.254:38401 requested a full tablet report, sending...
I20260927 07:01:14.128523 22047 ts_manager.cc:194] Registered new tserver with Master: 8859519f30c94e80b371d39982dacd82 (127.13.121.194:42779)
I20260927 07:01:14.129338 22047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:55841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:01:14.154722 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38401
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:14.478130 22375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:14.478444 22375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:14.478576 22375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:14.485944 22375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:14.486085 22375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:14.486234 22375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:01:14.496500 22375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:14.497869 22375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:14.499241 22375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:14.512439 22380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:14.512415 22381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:14.512379 22383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:14.513130 22375 server_base.cc:1034] running on GCE node
I20260927 07:01:14.513705 22375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:14.514806 22375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:14.516040 22375 hybrid_clock.cc:648] HybridClock initialized: now 1790492474516006 us; error 55 us; skew 500 ppm
I20260927 07:01:14.516379 22375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:14 dist-test-slave-p6hz krb5kdc[22002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492474, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:14.519601 22375 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:01:14.521209 22375 webserver.cc:460] Webserver started at http://127.13.121.195:39701/ using document root <none> and password file <none>
I20260927 07:01:14.521915 22375 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:14.522044 22375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:14.522341 22375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:14.525413 22375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "ae78949a4aa341b29c84e70f88540d41"
format_stamp: "Formatted at 2026-09-27 07:01:14 on dist-test-slave-p6hz"
server_key: "8b84db64955b815087561446ad449bde"
server_key_iv: "a11f53451d3c714eae821d62db33b627"
server_key_version: "encryptionkey@0"
I20260927 07:01:14.526186 22375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "ae78949a4aa341b29c84e70f88540d41"
format_stamp: "Formatted at 2026-09-27 07:01:14 on dist-test-slave-p6hz"
server_key: "8b84db64955b815087561446ad449bde"
server_key_iv: "a11f53451d3c714eae821d62db33b627"
server_key_version: "encryptionkey@0"
I20260927 07:01:14.532017 22375 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:01:14.536571 22390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:14.538489 22375 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 07:01:14.538681 22375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "ae78949a4aa341b29c84e70f88540d41"
format_stamp: "Formatted at 2026-09-27 07:01:14 on dist-test-slave-p6hz"
server_key: "8b84db64955b815087561446ad449bde"
server_key_iv: "a11f53451d3c714eae821d62db33b627"
server_key_version: "encryptionkey@0"
I20260927 07:01:14.538898 22375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:14.559463 22375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:14.563956 22375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:14.564240 22375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:14.565171 22375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:14.566520 22375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:14.566598 22375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:14.566717 22375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:14.566769 22375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:14.597607 22375 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:43869
I20260927 07:01:14.597632 22502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:43869 every 8 connection(s)
I20260927 07:01:14.598825 22375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:01:14.605671 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 22375
I20260927 07:01:14.605814 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:01:14.606216 13799 external_mini_cluster.cc:1428] Setting key a1aef14ebf71ab7aad7c3e6c876eb1f4
Sep 27 07:01:14 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492474, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:01:14.618041 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:14.601289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:36205 (local address 127.13.121.254:38401)
0927 07:01:14.601447 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:14.601452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:14.602558 (+ 1106us) server_negotiation.cc:407] Connection header received
0927 07:01:14.604373 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:14.604378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:14.604498 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:14.604592 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:14.606434 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.607305 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.608824 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.609223 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.609327 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:14.612735 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:14.612739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:14.612743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:14.612776 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:14.615087 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:14.615796 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:14.615802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:14.615805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:14.615930 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:14.616329 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:14.616333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:14.616335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:14.616604 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:14.616737 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:14.617474 (+ 737us) server_negotiation.cc:299] Negotiation successful
0927 07:01:14.617693 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 07:01:14.618968 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:14.601728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:36205)
0927 07:01:14.602404 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:14.602443 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:14.604113 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:14.604908 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:14.604921 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:14.605639 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:14.606254 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:14.606279 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.607454 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.607457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:14.608688 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:14.608701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.609433 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.609439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:14.609698 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:14.610365 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:14.610404 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:14.612527 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:14.615237 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:14.615247 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:14.615266 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:14.615683 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:14.616039 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:14.616043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:14.616047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:14.616229 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:14.616730 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:14.616737 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:14.617118 (+ 381us) client_negotiation.cc:769] Sending connection context
0927 07:01:14.617370 (+ 252us) client_negotiation.cc:240] Negotiation successful
0927 07:01:14.617909 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":254,"threads_started":1}
I20260927 07:01:14.620391 22503 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38401
I20260927 07:01:14.620687 22503 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:14.621486 22503 heartbeater.cc:511] Master 127.13.121.254:38401 requested a full tablet report, sending...
I20260927 07:01:14.622546 22236 heartbeater.cc:503] Master 127.13.121.254:38401 was elected leader, sending a full tablet report...
I20260927 07:01:14.623503 22046 ts_manager.cc:194] Registered new tserver with Master: ae78949a4aa341b29c84e70f88540d41 (127.13.121.195:43869)
I20260927 07:01:14.624370 22046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:36205
I20260927 07:01:14.633771 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab.
Entry for principal oryx/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab.
WARNING: no policy specified for oryx/127.13.121.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab.
Entry for principal oryx/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab.
WARNING: no policy specified for oryx/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab.
Entry for principal oryx/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab.
I20260927 07:01:14.768385 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22108
Sep 27 07:01:14 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:01:14.784430 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:14.769454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.13.121.193:42621)
0927 07:01:14.770076 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:14.770082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:14.770153 (+ 71us) server_negotiation.cc:407] Connection header received
0927 07:01:14.770312 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:14.770319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:14.770541 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:14.770665 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:14.772058 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.773094 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.774428 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.775017 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.775113 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:14.779049 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:14.779066 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:14.779082 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:14.779122 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:14.781780 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:14.782518 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:14.782526 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:14.782528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:14.782657 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:14.782978 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:14.782982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:14.782984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:14.783383 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:14.783602 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:14.783873 (+ 271us) server_negotiation.cc:299] Negotiation successful
0927 07:01:14.784074 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":298,"threads_started":1}
I20260927 07:01:14.785832 22231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:14.867828 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 22108
I20260927 07:01:14.881258 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22242
Sep 27 07:01:14 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:01:14.897233 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:14.882137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.13.121.194:42779)
0927 07:01:14.882638 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:14.882645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:14.882677 (+ 32us) server_negotiation.cc:407] Connection header received
0927 07:01:14.882743 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:14.882749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:14.882959 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:14.883102 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:14.884267 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.885273 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.886577 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.887125 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.887265 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:14.891506 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:14.891519 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:14.891534 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:14.891576 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:14.894334 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:14.895082 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:14.895090 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:14.895092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:14.895357 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:14.895772 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:14.895776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:14.895778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:14.896158 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:14.896427 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:14.896627 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 07:01:14.896827 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":248,"threads_started":1}
I20260927 07:01:14.898517 22365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:14.980000 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 22242
I20260927 07:01:14.993003 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22375
Sep 27 07:01:14 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:01:15.009696 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:14.993968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54180 (local address 127.13.121.195:43869)
0927 07:01:14.994602 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:14.994609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:14.994638 (+ 29us) server_negotiation.cc:407] Connection header received
0927 07:01:14.994696 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:14.994702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:14.994907 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:14.995029 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:14.996306 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.997332 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.998622 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:14.999437 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:14.999581 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:15.003646 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:15.003662 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:15.003682 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:15.003724 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:15.007229 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:15.007934 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:15.007941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:15.007944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:15.008067 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:15.008357 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:15.008360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:15.008362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:15.008692 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:15.008925 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:15.009137 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 07:01:15.009334 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":272,"threads_started":1}
I20260927 07:01:15.010983 22498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:15.092470 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 22375
I20260927 07:01:15.106122 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22015
I20260927 07:01:15.107304 22077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:15.207593 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 22015
I20260927 07:01:15.224743 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38401
--webserver_interface=127.13.121.254
--webserver_port=42395
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:38401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:15.547511 22520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:15.547820 22520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:15.547958 22520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:15.555343 22520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:01:15.555465 22520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:15.555534 22520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:15.555609 22520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:01:15.555670 22520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:01:15.565732 22520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:38401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=42395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:15.567085 22520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:15.568387 22520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:15.579927 22525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:15.579982 22526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:15.580001 22528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:15.580932 22520 server_base.cc:1034] running on GCE node
I20260927 07:01:15.581756 22520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:15.583317 22520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:15.584520 22520 hybrid_clock.cc:648] HybridClock initialized: now 1790492475584473 us; error 64 us; skew 500 ppm
I20260927 07:01:15.584858 22520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:15 dist-test-slave-p6hz krb5kdc[22002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492475, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:15.588008 22520 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:01:15.589680 22520 webserver.cc:460] Webserver started at http://127.13.121.254:42395/ using document root <none> and password file <none>
I20260927 07:01:15.590396 22520 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:15.590531 22520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:15.596644 22520 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260927 07:01:15.599659 22535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:15.601111 22520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:01:15.601284 22520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "27735d5fb8804135b8a1411744d1254c"
format_stamp: "Formatted at 2026-09-27 07:01:12 on dist-test-slave-p6hz"
server_key: "d86415e1245baea714edafcfb63e8de3"
server_key_iv: "375acfb1feda7c0735e2ce7698bdab6d"
server_key_version: "encryptionkey@0"
I20260927 07:01:15.602227 22520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:15.615653 22520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:15.619604 22520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:15.619858 22520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:15.636945 22520 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:38401
I20260927 07:01:15.636970 22586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:38401 every 8 connection(s)
I20260927 07:01:15.638417 22520 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:01:15.645238 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 22520
I20260927 07:01:15.645529 22587 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:01:15.646190 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:42621
--local_ip_for_outbound_sockets=127.13.121.193
--tserver_master_addrs=127.13.121.254:38401
--webserver_port=43045
--webserver_interface=127.13.121.193
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.121.193
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:01:15.650163 22587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: Bootstrap starting.
I20260927 07:01:15.669723 22587 log.cc:824] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:15.678613 22587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:01:15.679118 22587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: Bootstrap complete.
I20260927 07:01:15.687855 22587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } }
I20260927 07:01:15.688664 22587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27735d5fb8804135b8a1411744d1254c, State: Initialized, Role: FOLLOWER
I20260927 07:01:15.689257 22587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } }
I20260927 07:01:15.689471 22587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:01:15.689585 22587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:01:15.689738 22587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:01:15.693571 22587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } }
I20260927 07:01:15.694085 22587 leader_election.cc:302] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27735d5fb8804135b8a1411744d1254c; no voters:
I20260927 07:01:15.694811 22587 leader_election.cc:288] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 07:01:15.694945 22591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 2 FOLLOWER]: Leader election won for term 2
I20260927 07:01:15.696973 22591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [term 2 LEADER]: Becoming Leader. State: Replica: 27735d5fb8804135b8a1411744d1254c, State: Running, Role: LEADER
I20260927 07:01:15.697103 22587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:01:15.697552 22591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } }
I20260927 07:01:15.700258 22593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27735d5fb8804135b8a1411744d1254c. Latest consensus state: current_term: 2 leader_uuid: "27735d5fb8804135b8a1411744d1254c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } } }
I20260927 07:01:15.700646 22593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:15.700412 22592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "27735d5fb8804135b8a1411744d1254c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27735d5fb8804135b8a1411744d1254c" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38401 } } }
I20260927 07:01:15.701112 22592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:15.701872 22599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:01:15.716526 22599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:01:15.718010 22599 catalog_manager.cc:1259] Loaded cluster ID: 11c9012b4ceb4f31a04465299f6cf71b
I20260927 07:01:15.718163 22599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:01:15.722137 22599 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:01:15.723718 22599 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 27735d5fb8804135b8a1411744d1254c: Loaded TSK: 0
I20260927 07:01:15.725203 22599 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 07:01:15.978600 22589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:15.978899 22589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:15.979038 22589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:15.986521 22589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:15.986665 22589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:15.986814 22589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:01:15.996982 22589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.193.keytab
--principal=oryx/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:42621
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=43045
--tserver_master_addrs=127.13.121.254:38401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:15.998405 22589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:15.999795 22589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:16.015478 22613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:16.015607 22614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:16.015456 22616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:16.016793 22589 server_base.cc:1034] running on GCE node
I20260927 07:01:16.017412 22589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:16.018383 22589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:16.019578 22589 hybrid_clock.cc:648] HybridClock initialized: now 1790492476019516 us; error 55 us; skew 500 ppm
I20260927 07:01:16.019881 22589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492476, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:16.023226 22589 init.cc:377] Logged in from keytab as oryx/127.13.121.193@KRBTEST.COM (short username oryx)
I20260927 07:01:16.024775 22589 webserver.cc:460] Webserver started at http://127.13.121.193:43045/ using document root <none> and password file <none>
I20260927 07:01:16.025481 22589 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:16.025610 22589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:16.031368 22589 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 07:01:16.034229 22623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:16.035799 22589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:01:16.035964 22589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "cd0129f5d38b43e58a7eee84085cf65d"
format_stamp: "Formatted at 2026-09-27 07:01:13 on dist-test-slave-p6hz"
server_key: "731d5bd945020b7f0a4120a446dd6eee"
server_key_iv: "58b8dcdc7c4c069324281b0c30ae9a5f"
server_key_version: "encryptionkey@0"
I20260927 07:01:16.036950 22589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:16.049659 22589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:16.054107 22589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:16.054382 22589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:16.055454 22589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:16.056910 22589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:16.057013 22589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:16.057129 22589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:16.057197 22589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:16.087052 22589 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:42621
I20260927 07:01:16.087106 22735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:42621 every 8 connection(s)
I20260927 07:01:16.088526 22589 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:01:16.089583 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 22589
I20260927 07:01:16.090272 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:42779
--local_ip_for_outbound_sockets=127.13.121.194
--tserver_master_addrs=127.13.121.254:38401
--webserver_port=37317
--webserver_interface=127.13.121.194
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.121.194
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:16.110491 22738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:16.091799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:47823)
0927 07:01:16.092647 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:16.092695 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:16.094803 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:16.097210 (+ 2407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:16.097223 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:16.097990 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:16.098766 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.098791 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.101399 (+ 2608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.101404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.103052 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.103076 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.104296 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.104304 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.104573 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.105209 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:16.105261 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:16.109113 (+ 3852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":343,"threads_started":1}
W20260927 07:01:16.110935 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:16.091794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:47823 (local address 127.13.121.254:38401)
0927 07:01:16.095683 (+ 3889us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:16.095699 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:16.095746 (+ 47us) server_negotiation.cc:407] Connection header received
0927 07:01:16.096125 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:16.096144 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:16.096622 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:16.097026 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:16.099019 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.101142 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.103287 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.104032 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.104248 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.109477 (+ 5229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:47823: BlockingRecv error: recv got EOF from 127.13.121.193:47823 (error 108)
Metrics: {"server-negotiator.queue_time_us":3616,"thread_start_us":3396,"threads_started":1}
W20260927 07:01:16.111716 22736 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:38401 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.121.254:38401: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:16.120687 22738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:16.112349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:42773)
0927 07:01:16.112595 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:16.112610 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:16.112697 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:16.113001 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:16.113004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:16.113388 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:16.113531 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.113541 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.114972 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.114976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.115935 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.115949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.116854 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.116859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.116981 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.117561 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:16.117583 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:16.120474 (+ 2891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139}
W20260927 07:01:16.120975 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:16.112463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:42773 (local address 127.13.121.254:38401)
0927 07:01:16.112644 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:16.112650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:16.112672 (+ 22us) server_negotiation.cc:407] Connection header received
0927 07:01:16.112793 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:16.112797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:16.112879 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:16.112994 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:16.113677 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.114812 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.116073 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.116721 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.116843 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.120747 (+ 3904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:42773: BlockingRecv error: recv got EOF from 127.13.121.193:42773 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:16.129813 22738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:16.121376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:55603)
0927 07:01:16.121580 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:16.121600 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:16.121709 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:16.122260 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:16.122263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:16.122544 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:16.122711 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.122719 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.124189 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.124193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.125191 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.125205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.126049 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.126053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.126179 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.126794 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:16.126817 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:16.129579 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":536,"threads_started":1}
W20260927 07:01:16.130091 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:16.121621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:55603 (local address 127.13.121.254:38401)
0927 07:01:16.121810 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:16.121816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:16.121837 (+ 21us) server_negotiation.cc:407] Connection header received
0927 07:01:16.121908 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:16.121914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:16.122002 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:16.122099 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:16.122864 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.124033 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.125342 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.125926 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.126048 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.129839 (+ 3791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:55603: BlockingRecv error: recv got EOF from 127.13.121.193:55603 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260927 07:01:16.130204 22736 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 07:01:16.418290 22740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:16.418605 22740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:16.418722 22740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:16.426293 22740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:16.426430 22740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:16.426582 22740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:01:16.436646 22740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.194.keytab
--principal=oryx/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:42779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=37317
--tserver_master_addrs=127.13.121.254:38401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:16.437999 22740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:16.439311 22740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:16.453156 22746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:16.454245 22747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:16.454263 22749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:16.455262 22740 server_base.cc:1034] running on GCE node
I20260927 07:01:16.456007 22740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:16.457026 22740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:16.458266 22740 hybrid_clock.cc:648] HybridClock initialized: now 1790492476458217 us; error 42 us; skew 500 ppm
I20260927 07:01:16.458624 22740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492476, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:16.466378 22740 init.cc:377] Logged in from keytab as oryx/127.13.121.194@KRBTEST.COM (short username oryx)
I20260927 07:01:16.468039 22740 webserver.cc:460] Webserver started at http://127.13.121.194:37317/ using document root <none> and password file <none>
I20260927 07:01:16.468731 22740 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:16.468847 22740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:16.474870 22740 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260927 07:01:16.477761 22756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:16.479290 22740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:01:16.479449 22740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "8859519f30c94e80b371d39982dacd82"
format_stamp: "Formatted at 2026-09-27 07:01:14 on dist-test-slave-p6hz"
server_key: "5df5cc622eb0960998bdb6b60953c1d7"
server_key_iv: "bc46f249120071f546d1c32943b586ef"
server_key_version: "encryptionkey@0"
I20260927 07:01:16.480403 22740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:16.505306 22740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:16.509740 22740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:16.510005 22740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:16.511053 22740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:16.512460 22740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:16.512564 22740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:16.512668 22740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:16.512734 22740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:16.543251 22740 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:42779
I20260927 07:01:16.543300 22868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:42779 every 8 connection(s)
I20260927 07:01:16.544667 22740 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:01:16.545125 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 22740
I20260927 07:01:16.545902 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:43869
--local_ip_for_outbound_sockets=127.13.121.195
--tserver_master_addrs=127.13.121.254:38401
--webserver_port=39701
--webserver_interface=127.13.121.195
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.121.195
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 07:01:16.562211 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:16.547373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:57859 (local address 127.13.121.254:38401)
0927 07:01:16.547561 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:16.547566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:16.548891 (+ 1325us) server_negotiation.cc:407] Connection header received
0927 07:01:16.550746 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:16.550751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:16.550834 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:16.550919 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:16.552762 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.553474 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.555765 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.556179 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.556278 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.561986 (+ 5708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:57859: BlockingRecv error: recv got EOF from 127.13.121.194:57859 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:01:16.563282 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:16.547986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:57859)
0927 07:01:16.548727 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:16.548763 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:16.550443 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:16.551232 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:16.551242 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:16.551762 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:16.552558 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.552583 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.553648 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.553654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.555585 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.555615 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.556397 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.556403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.556679 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.557378 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:16.557418 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:16.561789 (+ 4371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":292,"threads_started":1}
W20260927 07:01:16.564100 22869 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:38401 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.121.254:38401: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:16.574268 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:16.564831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:40271)
0927 07:01:16.565043 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:16.565063 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:16.565163 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:16.566034 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:16.566040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:16.566507 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:16.566710 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.566724 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.567849 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.567855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.569296 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.569316 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.570019 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.570026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.570209 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.570800 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:16.570823 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:16.573994 (+ 3171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260927 07:01:16.574465 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:16.565296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:40271 (local address 127.13.121.254:38401)
0927 07:01:16.565652 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:16.565657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:16.565728 (+ 71us) server_negotiation.cc:407] Connection header received
0927 07:01:16.565801 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:16.565804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:16.565866 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:16.565944 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:16.566888 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.567693 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.569461 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.569862 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.569961 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.574236 (+ 4275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:40271: BlockingRecv error: recv got EOF from 127.13.121.194:40271 (error 108)
Metrics: {"server-negotiator.queue_time_us":219}
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:16.583334 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:16.574998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:60377)
0927 07:01:16.575260 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:16.575279 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:16.575362 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:16.575672 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:16.575675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:16.575983 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:16.576174 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.576186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.577605 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.577609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.578522 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.578533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.579511 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.579517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.579656 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.580223 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:16.580241 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:16.583033 (+ 2792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160}
W20260927 07:01:16.583567 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:16.575191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:60377 (local address 127.13.121.254:38401)
0927 07:01:16.575371 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:16.575377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:16.575425 (+ 48us) server_negotiation.cc:407] Connection header received
0927 07:01:16.575484 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:16.575489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:16.575560 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:16.575651 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:16.576319 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.577469 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.578649 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.579326 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.579461 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.583345 (+ 3884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:60377: BlockingRecv error: recv got EOF from 127.13.121.194:60377 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260927 07:01:16.583690 22869 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 07:01:16.875243 22872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:16.875558 22872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:16.875684 22872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:16.882898 22872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:16.883045 22872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:16.883239 22872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:01:16.895179 22872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/krb5kdc/oryx_127.13.121.195.keytab
--principal=oryx/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:43869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=39701
--tserver_master_addrs=127.13.121.254:38401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:16.896615 22872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:16.897974 22872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:16.910848 22877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:16.910754 22878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:16.910776 22880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:16.911506 22872 server_base.cc:1034] running on GCE node
I20260927 07:01:16.912071 22872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:16.912982 22872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:16.914201 22872 hybrid_clock.cc:648] HybridClock initialized: now 1790492476914142 us; error 43 us; skew 500 ppm
I20260927 07:01:16.914497 22872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492476, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:16.917501 22872 init.cc:377] Logged in from keytab as oryx/127.13.121.195@KRBTEST.COM (short username oryx)
I20260927 07:01:16.918934 22872 webserver.cc:460] Webserver started at http://127.13.121.195:39701/ using document root <none> and password file <none>
I20260927 07:01:16.919649 22872 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:16.919761 22872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:16.925364 22872 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 07:01:16.928151 22887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:16.929665 22872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:01:16.929816 22872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "ae78949a4aa341b29c84e70f88540d41"
format_stamp: "Formatted at 2026-09-27 07:01:14 on dist-test-slave-p6hz"
server_key: "8b84db64955b815087561446ad449bde"
server_key_iv: "a11f53451d3c714eae821d62db33b627"
server_key_version: "encryptionkey@0"
I20260927 07:01:16.930774 22872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:16.950388 22872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:16.954612 22872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:16.954840 22872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:16.955787 22872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:16.957073 22872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:16.957155 22872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:16.957238 22872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:16.957295 22872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:16.985939 22872 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:43869
I20260927 07:01:16.986006 22999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:43869 every 8 connection(s)
I20260927 07:01:16.987355 22872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:01:16.990212 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 22872
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:16 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 07:01:17.003844 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:16.989847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:45585 (local address 127.13.121.254:38401)
0927 07:01:16.990039 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:16.990043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:16.991555 (+ 1512us) server_negotiation.cc:407] Connection header received
0927 07:01:16.993752 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:16.993755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:16.993822 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:16.993892 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:16.995651 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.996365 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.997924 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.998407 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.998492 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.003598 (+ 5106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:45585: BlockingRecv error: recv got EOF from 127.13.121.195:45585 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260927 07:01:17.004662 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:16.990402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:45585)
0927 07:01:16.991330 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:16.991383 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:16.993430 (+ 2047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:16.994186 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:16.994199 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:16.994876 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:16.995482 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.995508 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.996520 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.996524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.997788 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:16.997800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.998637 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.998645 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:16.998944 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:16.999610 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:16.999647 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:17.003408 (+ 3761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":337,"threads_started":1}
W20260927 07:01:17.005319 23000 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:38401 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.121.254:38401: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
I20260927 07:01:17.006750 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:16.991275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52634 (local address 127.13.121.254:38401)
0927 07:01:16.992754 (+ 1479us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:16.992758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:16.992773 (+ 15us) server_negotiation.cc:407] Connection header received
0927 07:01:16.992821 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:16.992824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:16.992880 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:16.992960 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:16.994790 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.995579 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.996826 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:16.997252 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:16.997408 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.000394 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:17.000402 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:17.000418 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:17.000448 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:17.003444 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:17.004163 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:17.004172 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:17.004180 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:17.004379 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:17.004721 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:17.004726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:17.004729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:17.005335 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:17.005642 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:17.006123 (+ 481us) server_negotiation.cc:299] Negotiation successful
0927 07:01:17.006448 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1389,"thread_start_us":1293,"threads_started":1}
Sep 27 07:01:17 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:17 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:17.013675 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:17.005907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:54891)
0927 07:01:17.006146 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:17.006164 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:17.006314 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:17.006657 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:17.006662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:17.007030 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:17.007216 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:17.007228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.008239 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.008245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:17.009287 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:17.009302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.009986 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.009991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:17.010159 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.010660 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:17.010676 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:17.013356 (+ 2680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129}
W20260927 07:01:17.013778 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:17.005967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:54891 (local address 127.13.121.254:38401)
0927 07:01:17.006104 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:17.006109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:17.006226 (+ 117us) server_negotiation.cc:407] Connection header received
0927 07:01:17.006459 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:17.006464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:17.006541 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:17.006624 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:17.007344 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.008112 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.009415 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.009858 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.009959 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.013565 (+ 3606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:54891: BlockingRecv error: recv got EOF from 127.13.121.195:54891 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 27 07:01:17 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:17 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:17.022205 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:17.014333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:49485)
0927 07:01:17.014549 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:17.014569 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:17.014675 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:17.015115 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:17.015119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:17.015421 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:17.015682 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:17.015703 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.016742 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.016746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:17.017689 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:17.017701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.018373 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.018378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:17.018536 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.019078 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:17.019095 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:17.021920 (+ 2825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260927 07:01:17.022317 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:17.014538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:49485 (local address 127.13.121.254:38401)
0927 07:01:17.014783 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:17.014787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:17.014804 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:01:17.014870 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:17.014873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:17.014977 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:17.015077 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:17.015845 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.016600 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.017826 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.018230 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.018329 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.022083 (+ 3754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:49485: BlockingRecv error: recv got EOF from 127.13.121.195:49485 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
W20260927 07:01:17.022603 23000 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 07:01:17 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:17 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:17.139348 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:17.130875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:41355)
0927 07:01:17.131486 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:17.131502 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:17.131618 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:17.131952 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:17.131959 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:17.132415 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:17.132612 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:17.132622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.133626 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.133630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:17.134619 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:17.134634 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.135419 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.135427 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:17.135581 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.136176 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:17.136195 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:17.138978 (+ 2783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":386,"threads_started":1}
W20260927 07:01:17.139535 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:17.131049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:41355 (local address 127.13.121.254:38401)
0927 07:01:17.131286 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:17.131293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:17.131578 (+ 285us) server_negotiation.cc:407] Connection header received
0927 07:01:17.131713 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:17.131719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:17.131792 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:17.131884 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:17.132749 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.133478 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.134758 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.135248 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.135358 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.139329 (+ 3971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:41355: BlockingRecv error: recv got EOF from 127.13.121.193:41355 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Sep 27 07:01:17 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:17 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:17.592662 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:17.584290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:37867)
0927 07:01:17.584806 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:17.584824 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:17.584966 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:17.585279 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:17.585284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:17.585747 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:17.585932 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:17.585942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.586967 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.586974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:17.588039 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:17.588054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.588763 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.588770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:17.588914 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.589521 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:17.589539 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:17.592420 (+ 2881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":272,"threads_started":1}
W20260927 07:01:17.592955 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:17.584342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:37867 (local address 127.13.121.254:38401)
0927 07:01:17.584647 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:17.584653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:17.584902 (+ 249us) server_negotiation.cc:407] Connection header received
0927 07:01:17.585073 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:17.585078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:17.585150 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:17.585228 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:17.586105 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.586822 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.588180 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:17.588586 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:17.588693 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:17.592710 (+ 4017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:37867: BlockingRecv error: recv got EOF from 127.13.121.194:37867 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 07:01:18 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:18 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:18.032081 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:18.023379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:43103)
0927 07:01:18.023871 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:18.023887 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:18.024010 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:18.024391 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:18.024395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:18.024885 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:18.025126 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:18.025138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.026455 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.026460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:18.027509 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:18.027522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.028252 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.028257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:18.028374 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:18.028965 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:18.028985 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:18.031778 (+ 2793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":249,"threads_started":1}
W20260927 07:01:18.032272 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:18.023433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:43103 (local address 127.13.121.254:38401)
0927 07:01:18.023605 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:18.023610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:18.024060 (+ 450us) server_negotiation.cc:407] Connection header received
0927 07:01:18.024131 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:18.024135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:18.024199 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:18.024297 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:18.025279 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.026273 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.027698 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.028091 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.028194 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:18.032005 (+ 3811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:43103: BlockingRecv error: recv got EOF from 127.13.121.195:43103 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 07:01:18 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:18 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:18.148319 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:18.140371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:56223)
0927 07:01:18.140896 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:18.140915 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:18.141030 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:18.141284 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:18.141287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:18.141693 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:18.141864 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:18.141873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.142873 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.142889 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:18.143823 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:18.143834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.144628 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.144633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:18.144791 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:18.145366 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:18.145383 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:18.148072 (+ 2689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":286,"threads_started":1}
W20260927 07:01:18.148667 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:18.140446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:56223 (local address 127.13.121.254:38401)
0927 07:01:18.140662 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:18.140669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:18.140992 (+ 323us) server_negotiation.cc:407] Connection header received
0927 07:01:18.141123 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:18.141127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:18.141189 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:18.141270 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:18.141986 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.142711 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.143944 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.144477 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.144577 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:18.148278 (+ 3701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:56223: BlockingRecv error: recv got EOF from 127.13.121.193:56223 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 07:01:18 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:18 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:18.602366 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:18.593763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:32873)
0927 07:01:18.594255 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:18.594271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:18.594403 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:18.594746 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:18.594749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:18.595180 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:18.595389 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:18.595401 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.596512 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.596517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:18.597483 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:18.597501 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.598378 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.598383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:18.598509 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:18.599108 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:18.599127 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:18.602088 (+ 2961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":259,"threads_started":1}
W20260927 07:01:18.602600 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:18.593887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:32873 (local address 127.13.121.254:38401)
0927 07:01:18.594080 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:18.594085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:18.594368 (+ 283us) server_negotiation.cc:407] Connection header received
0927 07:01:18.594543 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:18.594548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:18.594613 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:18.594706 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:18.595534 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.596314 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.597638 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:18.598197 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:18.598307 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:18.602360 (+ 4053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:32873: BlockingRecv error: recv got EOF from 127.13.121.194:32873 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 07:01:19 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:19 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:19.041200 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:19.033063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:46523)
0927 07:01:19.033555 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:19.033572 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:19.033734 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:19.034051 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:19.034055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:19.034442 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:19.034615 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:19.034624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.035692 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.035698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:19.036590 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:19.036603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.037307 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.037313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:19.037471 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:19.038055 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:19.038072 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:19.040929 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":264,"threads_started":1}
W20260927 07:01:19.041445 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:19.033257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:46523 (local address 127.13.121.254:38401)
0927 07:01:19.033447 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:19.033453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:19.033645 (+ 192us) server_negotiation.cc:407] Connection header received
0927 07:01:19.033857 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:19.033863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:19.033921 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:19.033999 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:19.034749 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.035556 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.036726 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.037162 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.037267 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:19.041159 (+ 3892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:46523: BlockingRecv error: recv got EOF from 127.13.121.195:46523 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 27 07:01:19 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:19 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:19.157944 23011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:19.149291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:48025)
0927 07:01:19.149748 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:19.149768 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:19.149889 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:19.150256 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:19.150260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:19.150633 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:19.150802 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:19.150811 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.152201 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.152208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:19.153389 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:19.153405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.154097 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.154102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:19.154243 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:19.154842 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:19.154862 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:19.157682 (+ 2820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":252,"threads_started":1}
W20260927 07:01:19.158174 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:19.149443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:48025 (local address 127.13.121.254:38401)
0927 07:01:19.149680 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:19.149686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:19.149868 (+ 182us) server_negotiation.cc:407] Connection header received
0927 07:01:19.150024 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:19.150028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:19.150092 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:19.150176 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:19.150996 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.152031 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.153545 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.153934 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.154046 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:19.157928 (+ 3882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:48025: BlockingRecv error: recv got EOF from 127.13.121.193:48025 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 27 07:01:19 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:19 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:19.611338 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:19.603372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:53447)
0927 07:01:19.603861 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:19.603876 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:19.604011 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:19.604322 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:19.604327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:19.604691 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:19.604846 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:19.604855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.605886 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.605892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:19.606823 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:19.606835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.607639 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.607645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:19.607822 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:19.608378 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:19.608395 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:19.610978 (+ 2583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":264,"threads_started":1}
W20260927 07:01:19.611486 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:19.603507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:53447 (local address 127.13.121.254:38401)
0927 07:01:19.603720 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:19.603727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:19.603954 (+ 227us) server_negotiation.cc:407] Connection header received
0927 07:01:19.604128 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:19.604133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:19.604193 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:19.604278 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:19.605000 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.605760 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.606951 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:19.607485 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:19.607646 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:19.611264 (+ 3618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:53447: BlockingRecv error: recv got EOF from 127.13.121.194:53447 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 27 07:01:20 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:20 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:20.049997 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:20.042077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:57383)
0927 07:01:20.042553 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:20.042571 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:20.042709 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:20.043033 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:20.043036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:20.043406 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:20.043598 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:20.043607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.044582 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.044586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:20.045499 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:20.045511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.046174 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.046177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:20.046303 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:20.046910 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:20.046932 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:20.049737 (+ 2805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":226,"threads_started":1}
W20260927 07:01:20.050149 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:20.042208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:57383 (local address 127.13.121.254:38401)
0927 07:01:20.042409 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:20.042414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:20.042639 (+ 225us) server_negotiation.cc:407] Connection header received
0927 07:01:20.042809 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:20.042813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:20.042873 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:20.042954 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:20.043752 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.044449 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.045649 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.046051 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.046147 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:20.049957 (+ 3810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:57383: BlockingRecv error: recv got EOF from 127.13.121.195:57383 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 27 07:01:20 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:20 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:20.167232 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:20.158990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:39829)
0927 07:01:20.159524 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:20.159543 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:20.159677 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:20.160047 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:20.160053 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:20.160401 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:20.160545 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:20.160554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.161543 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.161549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:20.162569 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:20.162584 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.163541 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.163547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:20.163723 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:20.164324 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:20.164341 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:20.166950 (+ 2609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":270,"threads_started":1}
W20260927 07:01:20.167506 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:20.159248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:39829 (local address 127.13.121.254:38401)
0927 07:01:20.159484 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:20.159491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:20.159618 (+ 127us) server_negotiation.cc:407] Connection header received
0927 07:01:20.159802 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:20.159806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:20.159865 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:20.159944 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:20.160684 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.161408 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.162729 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.163379 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.163502 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:20.167244 (+ 3742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:39829: BlockingRecv error: recv got EOF from 127.13.121.193:39829 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 27 07:01:20 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:20 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:20.620334 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:20.612284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:43189)
0927 07:01:20.612749 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:20.612765 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:20.612901 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:20.613208 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:20.613213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:20.613578 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:20.613716 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:20.613726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.614683 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.614688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:20.615668 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:20.615681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.616391 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.616396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:20.616536 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:20.617106 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:20.617124 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:20.620113 (+ 2989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":258,"threads_started":1}
W20260927 07:01:20.620510 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:20.612371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:43189 (local address 127.13.121.254:38401)
0927 07:01:20.612587 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:20.612593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:20.612842 (+ 249us) server_negotiation.cc:407] Connection header received
0927 07:01:20.613013 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:20.613018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:20.613083 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:20.613163 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:20.613855 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.614559 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.615800 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:20.616224 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:20.616347 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:20.620277 (+ 3930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:43189: BlockingRecv error: recv got EOF from 127.13.121.194:43189 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 27 07:01:21 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:21 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:21.059597 23016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:21.050975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:55353)
0927 07:01:21.051688 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:21.051704 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:21.051824 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:21.052090 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:21.052093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:21.052421 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:21.052559 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:21.052567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.053697 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.053701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:21.054681 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:21.054695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.055498 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.055504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:21.055645 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:21.056318 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:21.056337 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:21.059301 (+ 2964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":498,"threads_started":1}
W20260927 07:01:21.059760 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:21.051023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:55353 (local address 127.13.121.254:38401)
0927 07:01:21.051239 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:21.051244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:21.051776 (+ 532us) server_negotiation.cc:407] Connection header received
0927 07:01:21.051919 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:21.051923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:21.051982 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:21.052070 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:21.052734 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.053527 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.054838 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.055314 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.055425 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:21.059545 (+ 4120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:55353: BlockingRecv error: recv got EOF from 127.13.121.195:55353 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 27 07:01:21 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:21 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:21.176986 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:21.168264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:55721)
0927 07:01:21.168804 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:21.168819 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:21.168955 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:21.169268 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:21.169273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:21.169702 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:21.169870 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:21.169879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.170915 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.170920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:21.171975 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:21.171989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.172710 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.172716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:21.172887 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:21.173516 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:21.173534 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:21.176737 (+ 3203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":266,"threads_started":1}
W20260927 07:01:21.177170 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:21.168375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:55721 (local address 127.13.121.254:38401)
0927 07:01:21.168622 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:21.168628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:21.168955 (+ 327us) server_negotiation.cc:407] Connection header received
0927 07:01:21.169079 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:21.169085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:21.169144 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:21.169225 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:21.170035 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.170770 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.172136 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.172563 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.172675 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:21.176928 (+ 4253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:55721: BlockingRecv error: recv got EOF from 127.13.121.193:55721 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 27 07:01:21 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:21 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:21.633059 23018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:21.621287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:60183)
0927 07:01:21.621796 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:21.621816 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:21.621955 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:21.622309 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:21.622314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:21.622742 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:21.622920 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:21.622929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.623987 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.623993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:21.624951 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:21.624967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.625880 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.625886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:21.626112 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:21.629815 (+ 3703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:21.629835 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:21.632787 (+ 2952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":231,"threads_started":1}
W20260927 07:01:21.633343 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:21.621576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:60183 (local address 127.13.121.254:38401)
0927 07:01:21.621795 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:21.621801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:21.621897 (+ 96us) server_negotiation.cc:407] Connection header received
0927 07:01:21.622102 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:21.622106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:21.622166 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:21.622250 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:21.623076 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.623840 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.625134 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:21.625727 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:21.625825 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:21.633070 (+ 7245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:60183: BlockingRecv error: recv got EOF from 127.13.121.194:60183 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 27 07:01:22 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:22 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:22.068801 23019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:22.060549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:36311)
0927 07:01:22.061094 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:22.061114 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:22.061242 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:22.061603 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:22.061608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:22.062043 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:22.062191 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:22.062200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.063226 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.063231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:22.064140 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:22.064153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.064855 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.064862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:22.065049 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:22.065627 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:22.065647 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:22.068479 (+ 2832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":259,"threads_started":1}
W20260927 07:01:22.068964 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:22.060778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:36311 (local address 127.13.121.254:38401)
0927 07:01:22.060936 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:22.060942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:22.061191 (+ 249us) server_negotiation.cc:407] Connection header received
0927 07:01:22.061376 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:22.061381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:22.061465 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:22.061545 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:22.062339 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.063051 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.064296 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.064714 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.064823 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:22.068746 (+ 3923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:36311: BlockingRecv error: recv got EOF from 127.13.121.195:36311 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 27 07:01:22 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:22 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:22.186506 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:22.177918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:45659)
0927 07:01:22.178453 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:22.178470 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:22.178601 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:22.178886 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:22.178889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:22.179330 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:22.179525 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:22.179536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.180540 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.180544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:22.181454 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:22.181466 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.182292 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.182298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:22.182432 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:22.183085 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:22.183102 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:22.186223 (+ 3121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":292,"threads_started":1}
W20260927 07:01:22.186725 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:22.178051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:45659 (local address 127.13.121.254:38401)
0927 07:01:22.178305 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:22.178311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:22.178553 (+ 242us) server_negotiation.cc:407] Connection header received
0927 07:01:22.178695 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:22.178699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:22.178759 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:22.178837 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:22.179669 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.180402 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.181627 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.182053 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.182156 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:22.186446 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:45659: BlockingRecv error: recv got EOF from 127.13.121.193:45659 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 27 07:01:22 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:22 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:22.642107 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:22.633959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:51193)
0927 07:01:22.634547 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:22.634565 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:22.634676 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:22.634942 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:22.634945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:22.635320 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:22.635482 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:22.635492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.636480 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.636484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:22.637393 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:22.637406 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.638078 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.638084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:22.638245 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:22.638781 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:22.638798 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:22.641852 (+ 3054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":319,"threads_started":1}
W20260927 07:01:22.642390 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:22.634042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:51193 (local address 127.13.121.254:38401)
0927 07:01:22.634196 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:22.634200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:22.634639 (+ 439us) server_negotiation.cc:407] Connection header received
0927 07:01:22.634760 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:22.634764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:22.634826 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:22.634913 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:22.635645 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.636342 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.637528 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:22.637934 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:22.638051 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:22.642139 (+ 4088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:51193: BlockingRecv error: recv got EOF from 127.13.121.194:51193 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 27 07:01:23 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:23 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:23.077906 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:23.069801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:39263)
0927 07:01:23.070253 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:23.070270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:23.070386 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:23.070679 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:23.070682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:23.071047 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:23.071244 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:23.071255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.072307 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.072312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:23.073236 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:23.073249 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.074034 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.074040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:23.074170 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:23.074756 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:23.074775 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:23.077649 (+ 2874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":226,"threads_started":1}
W20260927 07:01:23.078231 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:23.069906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:39263 (local address 127.13.121.254:38401)
0927 07:01:23.070169 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:23.070175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:23.070342 (+ 167us) server_negotiation.cc:407] Connection header received
0927 07:01:23.070486 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:23.070490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:23.070554 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:23.070636 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:23.071405 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.072160 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.073385 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.073830 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.073926 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:23.077899 (+ 3973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:39263: BlockingRecv error: recv got EOF from 127.13.121.195:39263 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Sep 27 07:01:23 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:23 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:23.196548 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:23.187556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:34271)
0927 07:01:23.188188 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:23.188205 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:23.188334 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:23.188619 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:23.188622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:23.189148 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:23.189328 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:23.189340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.190567 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.190571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:23.191675 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:23.191690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.192506 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.192520 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:23.192664 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:23.193298 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:23.193316 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:23.196294 (+ 2978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":293,"threads_started":1}
W20260927 07:01:23.196775 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:23.187669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:34271 (local address 127.13.121.254:38401)
0927 07:01:23.187889 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:23.187896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:23.188296 (+ 400us) server_negotiation.cc:407] Connection header received
0927 07:01:23.188432 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:23.188436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:23.188500 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:23.188580 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:23.189506 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.190413 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.191843 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.192267 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.192374 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:23.196533 (+ 4159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:34271: BlockingRecv error: recv got EOF from 127.13.121.193:34271 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 27 07:01:23 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:23 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:23.651422 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:23.643381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:52409)
0927 07:01:23.643949 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:23.643965 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:23.644111 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:23.644451 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:23.644457 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:23.644934 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:23.645153 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:23.645167 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.646115 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.646119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:23.647002 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:23.647013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.647692 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.647698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:23.647819 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:23.648408 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:23.648428 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:23.651157 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":237,"threads_started":1}
W20260927 07:01:23.651589 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:23.643427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:52409 (local address 127.13.121.254:38401)
0927 07:01:23.643654 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:23.643661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:23.644041 (+ 380us) server_negotiation.cc:407] Connection header received
0927 07:01:23.644232 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:23.644236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:23.644310 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:23.644386 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:23.645279 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.645985 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.647115 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:23.647530 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:23.647628 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:23.651357 (+ 3729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:52409: BlockingRecv error: recv got EOF from 127.13.121.194:52409 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 27 07:01:24 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:24 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:24.089059 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:24.079018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:56017)
0927 07:01:24.079593 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:24.079609 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:24.079742 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:24.080127 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:24.080132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:24.080527 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:24.080723 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:24.080733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.082072 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.082079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:24.083049 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:24.083063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.083810 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.083816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:24.084060 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:24.084758 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:24.084783 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:24.088734 (+ 3951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":238,"threads_started":1}
W20260927 07:01:24.089157 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:24.079119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:56017 (local address 127.13.121.254:38401)
0927 07:01:24.079326 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:24.079333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:24.079705 (+ 372us) server_negotiation.cc:407] Connection header received
0927 07:01:24.079877 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:24.079881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:24.079942 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:24.080020 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:24.080891 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.081923 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.083200 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.083652 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.083770 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:24.088961 (+ 5191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:56017: BlockingRecv error: recv got EOF from 127.13.121.195:56017 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 27 07:01:24 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:24 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:24.206158 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:24.197606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:46741)
0927 07:01:24.198175 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:24.198192 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:24.198328 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:24.198673 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:24.198679 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:24.199089 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:24.199287 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:24.199297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.200293 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.200300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:24.201224 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:24.201237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.202068 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.202076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:24.202223 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:24.202864 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:24.202882 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:24.205890 (+ 3008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":237,"threads_started":1}
W20260927 07:01:24.206408 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:24.197810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:46741 (local address 127.13.121.254:38401)
0927 07:01:24.198063 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:24.198069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:24.198270 (+ 201us) server_negotiation.cc:407] Connection header received
0927 07:01:24.198469 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:24.198474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:24.198547 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:24.198627 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:24.199423 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.200151 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.201364 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.201876 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.201989 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:24.206142 (+ 4153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:46741: BlockingRecv error: recv got EOF from 127.13.121.193:46741 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 27 07:01:24 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:24 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:24.660617 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:24.652441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:50929)
0927 07:01:24.652966 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:24.652983 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:24.653120 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:24.653432 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:24.653437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:24.653822 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:24.653964 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:24.653972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.655127 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.655162 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:24.656156 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:24.656169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.656924 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.656930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:24.657070 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:24.657645 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:24.657662 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:24.660380 (+ 2718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":282,"threads_started":1}
W20260927 07:01:24.660766 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:24.652577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:50929 (local address 127.13.121.254:38401)
0927 07:01:24.652775 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:24.652782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:24.653071 (+ 289us) server_negotiation.cc:407] Connection header received
0927 07:01:24.653235 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:24.653240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:24.653307 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:24.653386 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:24.654101 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.654986 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.656284 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:24.656753 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:24.656863 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:24.660579 (+ 3716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:50929: BlockingRecv error: recv got EOF from 127.13.121.194:50929 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 27 07:01:25 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:25 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:25.099550 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:25.091019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:35757)
0927 07:01:25.091585 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:25.091601 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:25.091732 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:25.092088 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:25.092092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:25.092456 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:25.092608 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:25.092618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.093630 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.093634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:25.094537 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:25.094550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.095285 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.095291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:25.095443 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:25.096212 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:25.096236 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:25.099216 (+ 2980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":294,"threads_started":1}
W20260927 07:01:25.099673 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:25.091130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:35757 (local address 127.13.121.254:38401)
0927 07:01:25.091439 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:25.091446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:25.091687 (+ 241us) server_negotiation.cc:407] Connection header received
0927 07:01:25.091859 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:25.091865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:25.091925 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:25.092009 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:25.092771 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.093481 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.094677 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.095091 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.095242 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:25.099417 (+ 4175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:35757: BlockingRecv error: recv got EOF from 127.13.121.195:35757 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 07:01:25 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:25 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:25.215806 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:25.207086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:35857)
0927 07:01:25.207655 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:25.207672 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:25.207796 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:25.208120 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:25.208123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:25.208516 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:25.208677 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:25.208686 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.209724 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.209729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:25.210996 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:25.211013 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.211790 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.211795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:25.211931 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:25.212575 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:25.212593 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:25.215464 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":247,"threads_started":1}
W20260927 07:01:25.215914 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:25.207187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:35857 (local address 127.13.121.254:38401)
0927 07:01:25.207400 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:25.207406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:25.207834 (+ 428us) server_negotiation.cc:407] Connection header received
0927 07:01:25.207891 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:25.207895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:25.208007 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:25.208091 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:25.208827 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.209586 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.211190 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.211621 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.211727 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:25.215705 (+ 3978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:35857: BlockingRecv error: recv got EOF from 127.13.121.193:35857 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 07:01:25 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:25 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:25.670207 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:25.661732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:46531)
0927 07:01:25.662421 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:25.662437 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:25.662585 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:25.662894 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:25.662899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:25.663334 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:25.663505 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:25.663515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.664509 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.664513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:25.665486 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:25.665507 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.666200 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.666207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:25.666372 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:25.666930 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:25.666947 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:25.669885 (+ 2938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":311,"threads_started":1}
W20260927 07:01:25.670413 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:25.662056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:46531 (local address 127.13.121.254:38401)
0927 07:01:25.662238 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:25.662244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:25.662530 (+ 286us) server_negotiation.cc:407] Connection header received
0927 07:01:25.662700 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:25.662704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:25.662764 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:25.662842 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:25.663645 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.664366 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.665635 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:25.666058 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:25.666163 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:25.670182 (+ 4019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:46531: BlockingRecv error: recv got EOF from 127.13.121.194:46531 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 27 07:01:26 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:26 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:26.110249 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:26.100461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:55781)
0927 07:01:26.100963 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:26.100982 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:26.101094 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:26.101472 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:26.101478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:26.101997 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:26.102232 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:26.102246 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.103460 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.103465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:26.104874 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:26.104892 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.106001 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.106010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:26.106230 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:26.106846 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:26.106865 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:26.109990 (+ 3125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":277,"threads_started":1}
W20260927 07:01:26.110399 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:26.100577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:55781 (local address 127.13.121.254:38401)
0927 07:01:26.100736 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:26.100740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:26.101056 (+ 316us) server_negotiation.cc:407] Connection header received
0927 07:01:26.101199 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:26.101203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:26.101277 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:26.101383 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:26.102397 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.103285 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.105103 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.105767 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.105904 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:26.110202 (+ 4298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:55781: BlockingRecv error: recv got EOF from 127.13.121.195:55781 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 27 07:01:26 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:26 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:26.225227 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:26.217048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:52291)
0927 07:01:26.217506 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:26.217522 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:26.217657 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:26.217956 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:26.217959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:26.218300 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:26.218446 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:26.218455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.219557 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.219561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:26.220482 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:26.220495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.221308 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.221314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:26.221467 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:26.222102 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:26.222121 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:26.224987 (+ 2866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":265,"threads_started":1}
W20260927 07:01:26.225467 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:26.217197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:52291 (local address 127.13.121.254:38401)
0927 07:01:26.217417 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:26.217422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:26.217622 (+ 200us) server_negotiation.cc:407] Connection header received
0927 07:01:26.217756 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:26.217760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:26.217837 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:26.217920 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:26.218625 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.219402 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.220645 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.221143 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.221264 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:26.225195 (+ 3931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:52291: BlockingRecv error: recv got EOF from 127.13.121.193:52291 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 27 07:01:26 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:26 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:26.686779 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:26.671353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:35327)
0927 07:01:26.671844 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:26.671862 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:26.672001 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:26.672305 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:26.672308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:26.672684 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:26.672831 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:26.672839 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.673768 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.673772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:26.674688 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:26.674701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.675467 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.675471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:26.675597 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:26.683763 (+ 8166us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:26.683785 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:26.686555 (+ 2770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":283,"threads_started":1}
W20260927 07:01:26.687043 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:26.671443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:35327 (local address 127.13.121.254:38401)
0927 07:01:26.671678 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:26.671684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:26.671989 (+ 305us) server_negotiation.cc:407] Connection header received
0927 07:01:26.672114 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:26.672118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:26.672181 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:26.672266 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:26.672946 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.673661 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.674827 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:26.675329 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:26.675424 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:26.686798 (+ 11374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:35327: BlockingRecv error: recv got EOF from 127.13.121.194:35327 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 27 07:01:27 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:27 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:27.120430 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:27.111668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:53479)
0927 07:01:27.112164 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:27.112182 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:27.112303 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:27.112633 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:27.112637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:27.112970 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:27.113157 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:27.113166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.114344 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.114350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:27.115671 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:27.115687 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.116463 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.116469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:27.116585 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:27.117237 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:27.117257 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:27.120189 (+ 2932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":249,"threads_started":1}
W20260927 07:01:27.120678 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:27.111922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:53479 (local address 127.13.121.254:38401)
0927 07:01:27.112223 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:27.112228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:27.112248 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:01:27.112417 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:27.112421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:27.112483 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:27.112579 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:27.113294 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.114174 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.115847 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.116274 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.116376 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:27.120388 (+ 4012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:53479: BlockingRecv error: recv got EOF from 127.13.121.195:53479 (error 108)
Metrics: {"server-negotiator.queue_time_us":184}
Sep 27 07:01:27 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:27 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:27.234493 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:27.226307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:36393)
0927 07:01:27.226804 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:27.226819 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:27.226957 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:27.227340 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:27.227344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:27.227760 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:27.227927 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:27.227936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.228977 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.228983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:27.229903 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:27.229916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.230622 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.230628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:27.230788 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:27.231413 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:27.231431 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:27.234229 (+ 2798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":279,"threads_started":1}
W20260927 07:01:27.234700 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:27.226421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:36393 (local address 127.13.121.254:38401)
0927 07:01:27.226658 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:27.226665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:27.226920 (+ 255us) server_negotiation.cc:407] Connection header received
0927 07:01:27.227084 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:27.227090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:27.227197 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:27.227282 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:27.228077 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.228838 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.230054 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.230460 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.230567 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:27.234491 (+ 3924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:36393: BlockingRecv error: recv got EOF from 127.13.121.193:36393 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Sep 27 07:01:27 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:27 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:27.696031 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:27.687771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:49755)
0927 07:01:27.688276 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:27.688293 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:27.688433 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:27.688799 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:27.688804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:27.689218 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:27.689388 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:27.689398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.690502 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.690507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:27.691439 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:27.691470 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.692223 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.692228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:27.692366 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:27.692916 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:27.692933 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:27.695787 (+ 2854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":273,"threads_started":1}
W20260927 07:01:27.696264 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:27.687871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:49755 (local address 127.13.121.254:38401)
0927 07:01:27.688050 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:27.688056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:27.688387 (+ 331us) server_negotiation.cc:407] Connection header received
0927 07:01:27.688556 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:27.688561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:27.688637 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:27.688735 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:27.689555 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.690314 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.691620 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:27.692053 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:27.692160 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:27.696029 (+ 3869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:49755: BlockingRecv error: recv got EOF from 127.13.121.194:49755 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 27 07:01:28 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:28 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:28.130053 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:28.121458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:48789)
0927 07:01:28.122003 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:28.122023 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:28.122179 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:28.122619 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:28.122624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:28.123187 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:28.123413 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:28.123426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.124715 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.124719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:28.125630 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:28.125641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.126352 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.126357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:28.126487 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:28.127108 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:28.127126 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:28.129810 (+ 2684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":284,"threads_started":1}
W20260927 07:01:28.130355 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:28.121642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:48789 (local address 127.13.121.254:38401)
0927 07:01:28.121853 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:28.121859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:28.122128 (+ 269us) server_negotiation.cc:407] Connection header received
0927 07:01:28.122317 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:28.122324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:28.122407 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:28.122522 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:28.123637 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.124581 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.125774 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.126189 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.126287 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:28.130021 (+ 3734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:48789: BlockingRecv error: recv got EOF from 127.13.121.195:48789 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 27 07:01:28 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:28 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:28.243683 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:28.235458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:36161)
0927 07:01:28.235963 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:28.235982 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:28.236108 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:28.236467 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:28.236473 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:28.236848 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:28.236996 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:28.237005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.238055 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.238060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:28.238970 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:28.238981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.239818 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.239824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:28.239966 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:28.240632 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:28.240659 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:28.243420 (+ 2761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":289,"threads_started":1}
W20260927 07:01:28.243818 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:28.235509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:36161 (local address 127.13.121.254:38401)
0927 07:01:28.235694 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:28.235701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:28.236046 (+ 345us) server_negotiation.cc:407] Connection header received
0927 07:01:28.236212 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:28.236215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:28.236298 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:28.236394 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:28.237151 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.237904 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.239114 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.239654 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.239765 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:28.243610 (+ 3845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:36161: BlockingRecv error: recv got EOF from 127.13.121.193:36161 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 27 07:01:28 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:28 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:28.705231 23039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:28.697012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:44825)
0927 07:01:28.697505 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:28.697522 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:28.697652 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:28.697937 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:28.697940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:28.698367 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:28.698518 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:28.698527 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.699512 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.699516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:28.700441 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:28.700453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.701267 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.701272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:28.701487 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:28.702055 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:28.702075 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:28.704949 (+ 2874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":275,"threads_started":1}
W20260927 07:01:28.705432 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:28.697085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:44825 (local address 127.13.121.254:38401)
0927 07:01:28.697305 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:28.697312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:28.697615 (+ 303us) server_negotiation.cc:407] Connection header received
0927 07:01:28.697743 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:28.697747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:28.697809 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:28.697885 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:28.698656 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.699378 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.700597 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:28.701079 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:28.701185 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:28.705195 (+ 4010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:44825: BlockingRecv error: recv got EOF from 127.13.121.194:44825 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 27 07:01:29 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:29 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:29.139509 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:29.131040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:47861)
0927 07:01:29.131593 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:29.131610 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:29.131761 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:29.132106 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:29.132111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:29.132527 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:29.132686 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:29.132695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.133725 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.133729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:29.134718 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:29.134732 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.135527 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.135534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:29.135719 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:29.136395 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:29.136418 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:29.139285 (+ 2867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":318,"threads_started":1}
W20260927 07:01:29.139771 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:29.131257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:47861 (local address 127.13.121.254:38401)
0927 07:01:29.131487 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:29.131493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:29.131715 (+ 222us) server_negotiation.cc:407] Connection header received
0927 07:01:29.131892 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:29.131898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:29.131961 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:29.132040 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:29.132836 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.133562 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.134866 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.135377 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.135490 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:29.139522 (+ 4032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:47861: BlockingRecv error: recv got EOF from 127.13.121.195:47861 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 27 07:01:29 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:29 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:29.253741 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:29.244743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:50367)
0927 07:01:29.245282 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:29.245298 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:29.245434 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:29.245784 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:29.245790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:29.246206 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:29.246360 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:29.246370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.247569 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.247575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:29.248692 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:29.248710 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.249550 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.249555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:29.249677 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:29.250452 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:29.250472 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:29.253500 (+ 3028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":269,"threads_started":1}
W20260927 07:01:29.253950 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:29.244898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:50367 (local address 127.13.121.254:38401)
0927 07:01:29.245135 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:29.245141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:29.245391 (+ 250us) server_negotiation.cc:407] Connection header received
0927 07:01:29.245571 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:29.245575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:29.245650 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:29.245733 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:29.246521 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.247409 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.248862 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.249353 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.249458 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:29.253738 (+ 4280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:50367: BlockingRecv error: recv got EOF from 127.13.121.193:50367 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 27 07:01:29 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:29 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:29.715377 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:29.706588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:57407)
0927 07:01:29.707263 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:29.707281 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:29.707400 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:29.707700 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:29.707703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:29.708082 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:29.708238 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:29.708247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.709340 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.709350 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:29.710317 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:29.710331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.711087 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.711091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:29.711303 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:29.711904 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:29.711921 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:29.715054 (+ 3133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":390,"threads_started":1}
W20260927 07:01:29.715631 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:29.706644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:57407 (local address 127.13.121.254:38401)
0927 07:01:29.706847 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:29.706854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:29.707357 (+ 503us) server_negotiation.cc:407] Connection header received
0927 07:01:29.707491 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:29.707495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:29.707558 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:29.707653 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:29.708419 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.709173 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.710495 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:29.710928 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:29.711032 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:29.715400 (+ 4368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:57407: BlockingRecv error: recv got EOF from 127.13.121.194:57407 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 27 07:01:30 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:30 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:30.149142 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:30.140684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:45507)
0927 07:01:30.141216 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:30.141235 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:30.141371 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:30.141735 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:30.141741 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:30.142111 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:30.142301 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:30.142315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.143622 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.143627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:30.144646 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:30.144660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.145365 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.145370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:30.145499 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:30.146125 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:30.146144 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:30.148878 (+ 2734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":284,"threads_started":1}
W20260927 07:01:30.149300 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:30.140710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:45507 (local address 127.13.121.254:38401)
0927 07:01:30.140896 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:30.140903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:30.141319 (+ 416us) server_negotiation.cc:407] Connection header received
0927 07:01:30.141512 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:30.141516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:30.141612 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:30.141707 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:30.142550 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.143424 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.144808 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.145223 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.145323 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:30.149114 (+ 3791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:45507: BlockingRecv error: recv got EOF from 127.13.121.195:45507 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 27 07:01:30 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:30 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:30.262660 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:30.254710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:56121)
0927 07:01:30.255213 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:30.255235 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:30.255345 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:30.255619 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:30.255622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:30.255951 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:30.256120 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:30.256129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.257217 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.257221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:30.258113 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:30.258125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.258833 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.258837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:30.258959 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:30.259583 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:30.259602 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:30.262389 (+ 2787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":257,"threads_started":1}
W20260927 07:01:30.262897 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:30.254792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:56121 (local address 127.13.121.254:38401)
0927 07:01:30.255043 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:30.255050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:30.255338 (+ 288us) server_negotiation.cc:407] Connection header received
0927 07:01:30.255429 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:30.255433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:30.255501 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:30.255582 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:30.256310 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.257061 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.258282 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.258701 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.258804 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:30.262642 (+ 3838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:56121: BlockingRecv error: recv got EOF from 127.13.121.193:56121 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Sep 27 07:01:30 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:30 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:30.724577 23045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:30.716373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:49419)
0927 07:01:30.716831 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:30.716849 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:30.716965 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:30.717245 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:30.717248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:30.717583 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:30.717757 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:30.717766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.718799 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.718804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:30.719764 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:30.719776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.720507 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.720513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:30.720672 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:30.721350 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:30.721371 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:30.724318 (+ 2947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":256,"threads_started":1}
W20260927 07:01:30.724704 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:30.716533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:49419 (local address 127.13.121.254:38401)
0927 07:01:30.716730 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:30.716734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:30.716924 (+ 190us) server_negotiation.cc:407] Connection header received
0927 07:01:30.717061 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:30.717064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:30.717123 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:30.717211 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:30.717924 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.718658 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.719916 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:30.720362 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:30.720468 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:30.724521 (+ 4053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:49419: BlockingRecv error: recv got EOF from 127.13.121.194:49419 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Sep 27 07:01:31 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:31 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.195@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:31.158524 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:31.150240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.195:36491)
0927 07:01:31.150752 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:31.150768 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:31.150903 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:31.151265 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:31.151270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:31.151617 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:31.151783 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:31.151792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.152894 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.152900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:31.153883 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:31.153896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.154596 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.154602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:31.154762 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:31.155429 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:31.155446 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:31.158291 (+ 2845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":275,"threads_started":1}
W20260927 07:01:31.158768 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:31.150362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:36491 (local address 127.13.121.254:38401)
0927 07:01:31.150576 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:31.150583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:31.150853 (+ 270us) server_negotiation.cc:407] Connection header received
0927 07:01:31.151028 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:31.151033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:31.151090 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:31.151209 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:31.151935 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.152754 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.154023 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.154451 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.154555 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:31.158524 (+ 3969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.195:36491: BlockingRecv error: recv got EOF from 127.13.121.195:36491 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 27 07:01:31 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:31 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.193@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:31.272753 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:31.263972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.193:45965)
0927 07:01:31.264727 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:31.264744 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:31.264862 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:31.265424 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:31.265429 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:31.265803 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:31.266015 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:31.266025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.267175 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.267181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:31.268129 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:31.268144 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.268947 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.268952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:31.269079 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:31.269704 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:31.269722 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:31.272493 (+ 2771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":496,"threads_started":1}
W20260927 07:01:31.272902 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:31.264010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:45965 (local address 127.13.121.254:38401)
0927 07:01:31.264228 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:31.264232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:31.264828 (+ 596us) server_negotiation.cc:407] Connection header received
0927 07:01:31.264982 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:31.264987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:31.265098 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:31.265186 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:31.266174 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.266940 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.268301 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.268770 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.268873 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:31.272707 (+ 3834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.193:45965: BlockingRecv error: recv got EOF from 127.13.121.193:45965 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 27 07:01:31 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 07:01:31 dist-test-slave-p6hz krb5kdc[22002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.121.194@KRBTEST.COM for oryx/127.13.121.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 07:01:31.737864 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:31.725749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38401 (local address 127.13.121.194:39497)
0927 07:01:31.726293 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:31.726309 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:31.726467 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:31.726793 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:31.726798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:31.727233 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:31.727393 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:31.727402 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.728420 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.728424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:31.729472 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:31.729488 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.730269 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.730274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:31.730409 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:31.734743 (+ 4334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:31.734768 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:31.737607 (+ 2839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.121.254:38401: Server oryx/127.13.121.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":262,"threads_started":1}
W20260927 07:01:31.738086 22739 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:31.725945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:39497 (local address 127.13.121.254:38401)
0927 07:01:31.726196 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:31.726203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:31.726402 (+ 199us) server_negotiation.cc:407] Connection header received
0927 07:01:31.726595 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:31.726599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:31.726657 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:31.726736 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:31.727551 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.728265 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.729599 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:31.730121 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:31.730250 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:31.737853 (+ 7603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.121.194:39497: BlockingRecv error: recv got EOF from 127.13.121.194:39497 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
I20260927 07:01:31.998216 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22589
I20260927 07:01:32.010314 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:31.999411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52004 (local address 127.13.121.193:42621)
0927 07:01:31.999991 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:31.999999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:32.000025 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:01:32.000084 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:32.000092 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:32.000308 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:32.000456 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:32.001971 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:32.003031 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:32.004262 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:32.004851 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:32.004954 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:32.007515 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:32.007523 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:32.007537 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:32.007593 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:32.009843 (+ 2250us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.121.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0927 07:01:32.010102 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52004: Request ticket server kudu/127.13.121.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":319,"threads_started":1}
W20260927 07:01:32.010402 23050 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52004: Request ticket server kudu/127.13.121.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260927 07:01:32.010613 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22589: Not authorized: Client connection negotiation failed: client connection to 127.13.121.193:42621: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.121.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260927 07:01:32.010823 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 22589
I20260927 07:01:32.024797 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22740
I20260927 07:01:32.037138 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:32.025901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44822 (local address 127.13.121.194:42779)
0927 07:01:32.026533 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:32.026545 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:32.026579 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:01:32.026641 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:32.026648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:32.026871 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:32.027024 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:32.028299 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:32.029338 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:32.030617 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:32.031346 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:32.031452 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:32.033935 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:32.033945 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:32.033960 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:32.034015 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:32.036610 (+ 2595us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.121.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0927 07:01:32.036901 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44822: Request ticket server kudu/127.13.121.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":330,"threads_started":1}
W20260927 07:01:32.037235 23051 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44822: Request ticket server kudu/127.13.121.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260927 07:01:32.037423 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22740: Not authorized: Client connection negotiation failed: client connection to 127.13.121.194:42779: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.121.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260927 07:01:32.037546 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 22740
I20260927 07:01:32.050626 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22872
I20260927 07:01:32.062637 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:32.051643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53138 (local address 127.13.121.195:43869)
0927 07:01:32.052215 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:32.052222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:32.052248 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:01:32.052308 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:32.052315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:32.052556 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:32.052713 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:32.054053 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:32.055185 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:32.056426 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:32.056993 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:32.057101 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:32.059762 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:32.059771 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:32.059786 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:32.059823 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:32.062126 (+ 2303us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.121.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0927 07:01:32.062402 (+ 276us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53138: Request ticket server kudu/127.13.121.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":318,"threads_started":1}
W20260927 07:01:32.062734 23052 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53138: Request ticket server kudu/127.13.121.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260927 07:01:32.062865 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22872: Not authorized: Client connection negotiation failed: client connection to 127.13.121.195:43869: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.13.121.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260927 07:01:32.063083 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 22872
I20260927 07:01:32.076143 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 22520
I20260927 07:01:32.077345 22582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:32.196247 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 22520
2026-09-27T07:01:32Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20848 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-27T07:01:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:01:32Z Disabled control of system clock
I20260927 07:01:32.266546 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:35427
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:44165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:35427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 07:01:32.589051 23057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:32.589346 23057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:32.589454 23057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:32.596805 23057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:01:32.596913 23057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:32.596971 23057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:32.597048 23057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:01:32.597107 23057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:01:32.606383 23057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:44165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:35427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:35427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:32.607733 23057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:32.609445 23057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:32.620972 23063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:32.621292 23065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:32.621028 23062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:32.621779 23057 server_base.cc:1034] running on GCE node
I20260927 07:01:32.622534 23057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:32.624032 23057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:32.625252 23057 hybrid_clock.cc:648] HybridClock initialized: now 1790492492625215 us; error 32 us; skew 500 ppm
I20260927 07:01:32.625607 23057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:01:32.627918 23057 webserver.cc:460] Webserver started at http://127.13.121.254:33237/ using document root <none> and password file <none>
I20260927 07:01:32.628582 23057 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:32.628669 23057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:32.628961 23057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:32.631824 23057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "adf8d4fa941645cb83931ab2a9278998"
format_stamp: "Formatted at 2026-09-27 07:01:32 on dist-test-slave-p6hz"
server_key: "3b1af797b2f4b0e08a050c388eb1d807"
server_key_iv: "9f949fc5e45e7b8a417e09e7c2be18db"
server_key_version: "encryptionkey@0"
I20260927 07:01:32.632571 23057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "adf8d4fa941645cb83931ab2a9278998"
format_stamp: "Formatted at 2026-09-27 07:01:32 on dist-test-slave-p6hz"
server_key: "3b1af797b2f4b0e08a050c388eb1d807"
server_key_iv: "9f949fc5e45e7b8a417e09e7c2be18db"
server_key_version: "encryptionkey@0"
I20260927 07:01:32.637540 23057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:01:32.640906 23071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:32.642186 23057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:01:32.642333 23057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "adf8d4fa941645cb83931ab2a9278998"
format_stamp: "Formatted at 2026-09-27 07:01:32 on dist-test-slave-p6hz"
server_key: "3b1af797b2f4b0e08a050c388eb1d807"
server_key_iv: "9f949fc5e45e7b8a417e09e7c2be18db"
server_key_version: "encryptionkey@0"
I20260927 07:01:32.642529 23057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:32.666158 23057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:32.667070 23057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:32.667380 23057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:32.684619 23057 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:35427
I20260927 07:01:32.684655 23122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:35427 every 8 connection(s)
I20260927 07:01:32.685917 23057 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:01:32.687867 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 23057
I20260927 07:01:32.688033 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:01:32.688654 13799 external_mini_cluster.cc:1428] Setting key 1130ddbd98de9acaa02f2612a49bf22d
I20260927 07:01:32.689826 23123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:32.701891 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:32.690580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.13.121.254:35427)
0927 07:01:32.691603 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:32.691614 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:32.691666 (+ 52us) server_negotiation.cc:407] Connection header received
0927 07:01:32.693189 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:32.693213 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:32.693729 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:32.694123 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:32.694713 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:32.696090 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:32.697388 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:32.698031 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:32.698189 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:32.699618 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:32.699629 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:32.699644 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:32.700225 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:32.700301 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:32.700325 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:32.700432 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:32.700663 (+ 231us) server_negotiation.cc:299] Negotiation successful
0927 07:01:32.700958 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":356,"threads_started":1}
I20260927 07:01:32.702214 23123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998: Bootstrap starting.
I20260927 07:01:32.705773 23123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:32.706744 23123 log.cc:824] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:32.709599 23123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998: No bootstrap required, opened a new log
I20260927 07:01:32.716816 23123 raft_consensus.cc:348] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf8d4fa941645cb83931ab2a9278998" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35427 } }
I20260927 07:01:32.717185 23123 raft_consensus.cc:374] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:32.717263 23123 raft_consensus.cc:729] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf8d4fa941645cb83931ab2a9278998, State: Initialized, Role: FOLLOWER
I20260927 07:01:32.717765 23123 consensus_queue.cc:260] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf8d4fa941645cb83931ab2a9278998" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35427 } }
I20260927 07:01:32.717959 23123 raft_consensus.cc:388] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:01:32.718053 23123 raft_consensus.cc:482] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:01:32.718183 23123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:32.720309 23123 raft_consensus.cc:504] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf8d4fa941645cb83931ab2a9278998" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35427 } }
I20260927 07:01:32.720817 23123 leader_election.cc:302] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adf8d4fa941645cb83931ab2a9278998; no voters:
I20260927 07:01:32.721370 23123 leader_election.cc:288] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:01:32.721508 23128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:32.722597 23128 raft_consensus.cc:686] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [term 1 LEADER]: Becoming Leader. State: Replica: adf8d4fa941645cb83931ab2a9278998, State: Running, Role: LEADER
I20260927 07:01:32.723253 23128 consensus_queue.cc:237] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf8d4fa941645cb83931ab2a9278998" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35427 } }
I20260927 07:01:32.723582 23123 sys_catalog.cc:564] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:01:32.726084 23129 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adf8d4fa941645cb83931ab2a9278998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf8d4fa941645cb83931ab2a9278998" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35427 } } }
I20260927 07:01:32.726075 23130 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adf8d4fa941645cb83931ab2a9278998. Latest consensus state: current_term: 1 leader_uuid: "adf8d4fa941645cb83931ab2a9278998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf8d4fa941645cb83931ab2a9278998" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 35427 } } }
I20260927 07:01:32.726500 23130 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:32.726505 23129 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998 [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:32.727380 23136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:01:32.733557 23136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:01:32.742566 23136 catalog_manager.cc:1347] Generated new cluster ID: e2c3523d8a6e410a9a5b004b6e9ddfc7
I20260927 07:01:32.742710 23136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:01:32.764786 23136 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:01:32.766214 23136 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:01:32.778991 23136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adf8d4fa941645cb83931ab2a9278998: Generated new TSK 0
I20260927 07:01:32.779908 23136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:01:32.786319 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35427
--builtin_ntp_servers=127.13.121.212:44165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 07:01:33.112000 23147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:33.112304 23147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:33.112407 23147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:33.120188 23147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:33.120312 23147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:33.120468 23147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:01:33.130715 23147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:44165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:33.132122 23147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:33.133978 23147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:33.147771 23155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:33.147698 23152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:33.147765 23153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:33.148444 23147 server_base.cc:1034] running on GCE node
I20260927 07:01:33.149070 23147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:33.150174 23147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:33.151458 23147 hybrid_clock.cc:648] HybridClock initialized: now 1790492493151396 us; error 57 us; skew 500 ppm
I20260927 07:01:33.151803 23147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:01:33.154400 23147 webserver.cc:460] Webserver started at http://127.13.121.193:33803/ using document root <none> and password file <none>
I20260927 07:01:33.155395 23147 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:33.155550 23147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:33.156055 23147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:33.159617 23147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "5bb540dd91a4490793b8a0177a0e9c15"
format_stamp: "Formatted at 2026-09-27 07:01:33 on dist-test-slave-p6hz"
server_key: "e621de61627c5ea26c0ba678efea4dc1"
server_key_iv: "2e1f4da359275d55f9a7ba1889babdcc"
server_key_version: "encryptionkey@0"
I20260927 07:01:33.160701 23147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "5bb540dd91a4490793b8a0177a0e9c15"
format_stamp: "Formatted at 2026-09-27 07:01:33 on dist-test-slave-p6hz"
server_key: "e621de61627c5ea26c0ba678efea4dc1"
server_key_iv: "2e1f4da359275d55f9a7ba1889babdcc"
server_key_version: "encryptionkey@0"
I20260927 07:01:33.167400 23147 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 07:01:33.171372 23161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:33.172922 23147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:01:33.173153 23147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "5bb540dd91a4490793b8a0177a0e9c15"
format_stamp: "Formatted at 2026-09-27 07:01:33 on dist-test-slave-p6hz"
server_key: "e621de61627c5ea26c0ba678efea4dc1"
server_key_iv: "2e1f4da359275d55f9a7ba1889babdcc"
server_key_version: "encryptionkey@0"
I20260927 07:01:33.173374 23147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:33.197208 23147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:33.198099 23147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:33.198393 23147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:33.199420 23147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:33.200953 23147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:33.201046 23147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:33.201180 23147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:33.201257 23147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:33.231822 23147 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:36509
I20260927 07:01:33.231899 23273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:36509 every 8 connection(s)
I20260927 07:01:33.233141 23147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:01:33.242205 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 23147
I20260927 07:01:33.242378 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:01:33.243000 13799 external_mini_cluster.cc:1428] Setting key cc0bf44b4856748846218c52c5c067eb
I20260927 07:01:33.245640 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35427
--builtin_ntp_servers=127.13.121.212:44165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 07:01:33.246017 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:33.235820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:54661 (local address 127.13.121.254:35427)
0927 07:01:33.236240 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:33.236245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:33.237458 (+ 1213us) server_negotiation.cc:407] Connection header received
0927 07:01:33.239238 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:33.239244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:33.239327 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:33.239420 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:33.240604 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:33.241333 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:33.243026 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:33.243508 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:33.243624 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:33.245201 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:33.245208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:33.245212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:33.245234 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:33.245278 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:33.245292 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:33.245364 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:33.245596 (+ 232us) server_negotiation.cc:299] Negotiation successful
0927 07:01:33.245692 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":228,"threads_started":1}
I20260927 07:01:33.247123 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:33.236295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:35427 (local address 127.13.121.193:54661)
0927 07:01:33.237279 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:33.237322 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:33.238933 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:33.239685 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:33.239694 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:33.239795 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:33.240422 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:33.240440 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:33.241497 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:33.241502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:33.242873 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:33.242886 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:33.243740 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:33.243747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:33.243980 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:33.244818 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:01:33.244820 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:33.244862 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:01:33.244950 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:33.245383 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:33.245393 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:33.245401 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:01:33.245565 (+ 164us) client_negotiation.cc:240] Negotiation successful
0927 07:01:33.246038 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":278,"threads_started":1}
I20260927 07:01:33.248631 23274 heartbeater.cc:347] Connected to a master server at 127.13.121.254:35427
I20260927 07:01:33.248919 23274 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:33.249679 23274 heartbeater.cc:511] Master 127.13.121.254:35427 requested a full tablet report, sending...
I20260927 07:01:33.252027 23088 ts_manager.cc:194] Registered new tserver with Master: 5bb540dd91a4490793b8a0177a0e9c15 (127.13.121.193:36509)
I20260927 07:01:33.253553 23088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.193:54661
W20260927 07:01:33.572360 23278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:33.572665 23278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:33.572752 23278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:33.580192 23278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:33.580312 23278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:33.580416 23278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:01:33.589687 23278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:44165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:33.590950 23278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:33.592720 23278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:33.605336 23283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:33.605391 23286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:33.605369 23284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:33.605998 23278 server_base.cc:1034] running on GCE node
I20260927 07:01:33.606623 23278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:33.607652 23278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:33.608867 23278 hybrid_clock.cc:648] HybridClock initialized: now 1790492493608816 us; error 51 us; skew 500 ppm
I20260927 07:01:33.609200 23278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:01:33.611402 23278 webserver.cc:460] Webserver started at http://127.13.121.194:40671/ using document root <none> and password file <none>
I20260927 07:01:33.612083 23278 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:33.612180 23278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:33.612481 23278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:33.615254 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "f83bbfb96f3f4584a86cbfdc3fe8a3b2"
format_stamp: "Formatted at 2026-09-27 07:01:33 on dist-test-slave-p6hz"
server_key: "3cf428710a25d9e9795e3fc437c00327"
server_key_iv: "320b154175020ebf9b8e540b29fff32f"
server_key_version: "encryptionkey@0"
I20260927 07:01:33.616041 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "f83bbfb96f3f4584a86cbfdc3fe8a3b2"
format_stamp: "Formatted at 2026-09-27 07:01:33 on dist-test-slave-p6hz"
server_key: "3cf428710a25d9e9795e3fc437c00327"
server_key_iv: "320b154175020ebf9b8e540b29fff32f"
server_key_version: "encryptionkey@0"
I20260927 07:01:33.621100 23278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260927 07:01:33.624732 23292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:33.626093 23278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:01:33.626274 23278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "f83bbfb96f3f4584a86cbfdc3fe8a3b2"
format_stamp: "Formatted at 2026-09-27 07:01:33 on dist-test-slave-p6hz"
server_key: "3cf428710a25d9e9795e3fc437c00327"
server_key_iv: "320b154175020ebf9b8e540b29fff32f"
server_key_version: "encryptionkey@0"
I20260927 07:01:33.626475 23278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:33.658334 23278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:33.659277 23278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:33.659554 23278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:33.660501 23278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:33.661929 23278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:33.662014 23278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:33.662123 23278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:33.662209 23278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:33.693359 23278 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:32807
I20260927 07:01:33.693411 23404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:32807 every 8 connection(s)
I20260927 07:01:33.694669 23278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:01:33.700116 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 23278
I20260927 07:01:33.700281 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:01:33.700804 13799 external_mini_cluster.cc:1428] Setting key 16de025b200ff3c3537415ee1dea290d
I20260927 07:01:33.703608 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35427
--builtin_ntp_servers=127.13.121.212:44165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 07:01:33.708619 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:33.697380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:57329 (local address 127.13.121.254:35427)
0927 07:01:33.697550 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:33.697555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:33.699082 (+ 1527us) server_negotiation.cc:407] Connection header received
0927 07:01:33.701316 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:33.701320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:33.701402 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:33.701517 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:33.702752 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:33.703526 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:33.705873 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:33.706244 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:33.706342 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:33.707902 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:33.707907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:33.707910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:33.707930 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:33.707960 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:33.707972 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:33.708041 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:33.708278 (+ 237us) server_negotiation.cc:299] Negotiation successful
0927 07:01:33.708366 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 07:01:33.710006 23407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:33.697996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:35427 (local address 127.13.121.194:57329)
0927 07:01:33.698914 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:33.698954 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:33.700992 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:33.701760 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:33.701772 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:33.701893 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:33.702572 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:33.702599 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:33.703701 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:33.703706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:33.705701 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:33.705717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:33.706468 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:33.706474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:33.706750 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:33.707536 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:01:33.707540 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:33.707597 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:01:33.707697 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:33.708061 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:33.708070 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:33.708077 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 07:01:33.708266 (+ 189us) client_negotiation.cc:240] Negotiation successful
0927 07:01:33.708689 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":295,"threads_started":1}
I20260927 07:01:33.711621 23405 heartbeater.cc:347] Connected to a master server at 127.13.121.254:35427
I20260927 07:01:33.711908 23405 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:33.712688 23405 heartbeater.cc:511] Master 127.13.121.254:35427 requested a full tablet report, sending...
I20260927 07:01:33.714165 23088 ts_manager.cc:194] Registered new tserver with Master: f83bbfb96f3f4584a86cbfdc3fe8a3b2 (127.13.121.194:32807)
I20260927 07:01:33.715237 23088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.194:57329
W20260927 07:01:34.030524 23408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:34.030829 23408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:34.030915 23408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:34.038385 23408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:34.038502 23408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:34.038642 23408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:01:34.048193 23408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:44165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:35427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:34.049479 23408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:34.051364 23408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:34.065254 23414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:34.065315 23413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:34.065315 23416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:34.065984 23408 server_base.cc:1034] running on GCE node
I20260927 07:01:34.066659 23408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:34.067699 23408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:34.068936 23408 hybrid_clock.cc:648] HybridClock initialized: now 1790492494068881 us; error 48 us; skew 500 ppm
I20260927 07:01:34.069326 23408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:01:34.071766 23408 webserver.cc:460] Webserver started at http://127.13.121.195:33141/ using document root <none> and password file <none>
I20260927 07:01:34.072494 23408 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:34.072599 23408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:34.072908 23408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:34.075801 23408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "18bed896a6b34eee91caa91e07df0153"
format_stamp: "Formatted at 2026-09-27 07:01:34 on dist-test-slave-p6hz"
server_key: "5cf289c595b85e2dcf79ddd1934a4534"
server_key_iv: "411ad99e5f02872a71968fa63c984714"
server_key_version: "encryptionkey@0"
I20260927 07:01:34.076607 23408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "18bed896a6b34eee91caa91e07df0153"
format_stamp: "Formatted at 2026-09-27 07:01:34 on dist-test-slave-p6hz"
server_key: "5cf289c595b85e2dcf79ddd1934a4534"
server_key_iv: "411ad99e5f02872a71968fa63c984714"
server_key_version: "encryptionkey@0"
I20260927 07:01:34.082137 23408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:01:34.086006 23422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:34.087718 23408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:01:34.087909 23408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "18bed896a6b34eee91caa91e07df0153"
format_stamp: "Formatted at 2026-09-27 07:01:34 on dist-test-slave-p6hz"
server_key: "5cf289c595b85e2dcf79ddd1934a4534"
server_key_iv: "411ad99e5f02872a71968fa63c984714"
server_key_version: "encryptionkey@0"
I20260927 07:01:34.088129 23408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:34.103098 23408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:34.104369 23408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:34.104773 23408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:34.106189 23408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:34.108129 23408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:34.108245 23408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:34.108361 23408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:34.108465 23408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260927 07:01:34.142071 23408 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:40199
I20260927 07:01:34.142122 23534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:40199 every 8 connection(s)
I20260927 07:01:34.143613 23408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:01:34.148386 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 23408
I20260927 07:01:34.148527 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:01:34.148923 13799 external_mini_cluster.cc:1428] Setting key 76d8a3efbf927407e553f7fbb9606f1e
I20260927 07:01:34.156569 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:34.146021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:33273 (local address 127.13.121.254:35427)
0927 07:01:34.146214 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:34.146220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:34.147751 (+ 1531us) server_negotiation.cc:407] Connection header received
0927 07:01:34.149499 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:34.149505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:34.149592 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:34.149671 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:34.150899 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.151975 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.153642 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.154050 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.154169 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:34.155752 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:34.155758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:34.155761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:34.155781 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:34.155811 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:34.155824 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:34.155900 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:34.156155 (+ 255us) server_negotiation.cc:299] Negotiation successful
0927 07:01:34.156256 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:01:34.157641 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:34.146576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:35427 (local address 127.13.121.195:33273)
0927 07:01:34.147598 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:34.147641 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:34.149221 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:34.149930 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:34.149938 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:34.150116 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:34.150712 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:34.150736 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.152174 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.152178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:34.153479 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:34.153491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.154278 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.154283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:34.154492 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:34.155366 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:01:34.155369 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:34.155418 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:01:34.155504 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:34.155930 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:34.155941 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:34.155950 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 07:01:34.156121 (+ 171us) client_negotiation.cc:240] Negotiation successful
0927 07:01:34.156558 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":345,"threads_started":1}
I20260927 07:01:34.159207 23535 heartbeater.cc:347] Connected to a master server at 127.13.121.254:35427
I20260927 07:01:34.159488 23535 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:34.160260 23535 heartbeater.cc:511] Master 127.13.121.254:35427 requested a full tablet report, sending...
I20260927 07:01:34.161762 23088 ts_manager.cc:194] Registered new tserver with Master: 18bed896a6b34eee91caa91e07df0153 (127.13.121.195:40199)
I20260927 07:01:34.162614 23088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.195:33273
I20260927 07:01:34.164507 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 07:01:34.170468 23276 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:34.168780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.13.121.254:35427)
0927 07:01:34.168934 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:34.168939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:34.169428 (+ 489us) server_negotiation.cc:407] Connection header received
0927 07:01:34.169630 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:34.169635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:34.169717 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:34.169811 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:34.170262 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54074: BlockingRecv error: recv got EOF from 127.0.0.1:54074 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 07:01:34.171085 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 23147
I20260927 07:01:34.178578 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:34.172069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51532 (local address 127.13.121.193:36509)
0927 07:01:34.172626 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:34.172640 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:34.172665 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:01:34.172718 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:34.172724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:34.172928 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:34.173045 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:34.173368 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.174345 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.175663 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.176202 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.176301 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:34.177223 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:34.177234 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:34.177252 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:34.177875 (+ 623us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:34.177919 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:34.177940 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:34.178024 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:34.178165 (+ 141us) server_negotiation.cc:299] Negotiation successful
0927 07:01:34.178268 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":302,"threads_started":1}
I20260927 07:01:34.179795 23269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:34.261274 23274 heartbeater.cc:503] Master 127.13.121.254:35427 was elected leader, sending a full tablet report...
I20260927 07:01:34.262228 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 23147
W20260927 07:01:34.275662 23072 connection.cc:535] server connection from 127.13.121.193:54661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 07:01:34.275936 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 23278
I20260927 07:01:34.283515 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:34.276792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43782 (local address 127.13.121.194:32807)
0927 07:01:34.277387 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:34.277394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:34.277422 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:01:34.277485 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:34.277492 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:34.277708 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:34.277830 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:34.278160 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.279167 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.280390 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.280958 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.281092 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:34.282040 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:34.282049 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:34.282063 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:34.282708 (+ 645us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:34.282754 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:34.282773 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:34.282872 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:34.283002 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 07:01:34.283172 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":325,"threads_started":1}
I20260927 07:01:34.284705 23400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:34.364714 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 23278
I20260927 07:01:34.377564 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 23408
I20260927 07:01:34.385434 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:34.378591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48630 (local address 127.13.121.195:40199)
0927 07:01:34.379259 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:34.379266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:34.379296 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:01:34.379352 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:34.379358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:34.379571 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:34.379697 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:34.380108 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.381109 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.382267 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:34.382881 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:34.383008 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:34.384110 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:34.384119 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:34.384133 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:34.384698 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:34.384743 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:34.384763 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:34.384858 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:34.384975 (+ 117us) server_negotiation.cc:299] Negotiation successful
0927 07:01:34.385134 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":322,"threads_started":1}
I20260927 07:01:34.386665 23530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:34.468029 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 23408
I20260927 07:01:34.480615 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 23057
I20260927 07:01:34.481860 23118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:34.582384 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 23057
2026-09-27T07:01:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2382 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-27T07:01:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:01:34Z Disabled control of system clock
I20260927 07:01:34.649686 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:39783
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:39783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 07:01:34.974838 23554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:34.975178 23554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:34.975296 23554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:34.982730 23554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:01:34.982841 23554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:34.982879 23554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:34.982988 23554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:01:34.983045 23554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:01:34.992542 23554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:39783
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:39783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:34.993837 23554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:34.995647 23554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:35.007182 23562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:35.007244 23560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:35.007263 23559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:35.007928 23554 server_base.cc:1034] running on GCE node
I20260927 07:01:35.008764 23554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:35.010213 23554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:35.011413 23554 hybrid_clock.cc:648] HybridClock initialized: now 1790492495011370 us; error 32 us; skew 500 ppm
I20260927 07:01:35.011749 23554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:01:35.014052 23554 webserver.cc:460] Webserver started at http://127.13.121.254:41619/ using document root <none> and password file <none>
I20260927 07:01:35.014806 23554 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:35.014910 23554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:35.015288 23554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:35.018128 23554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "b71ce1c68f184b739625b05a6953f87e"
format_stamp: "Formatted at 2026-09-27 07:01:35 on dist-test-slave-p6hz"
server_key: "9f29986baea22b9dff18d590d07cc052"
server_key_iv: "f2d55be595c2becf215c425034594ef2"
server_key_version: "encryptionkey@0"
I20260927 07:01:35.018898 23554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "b71ce1c68f184b739625b05a6953f87e"
format_stamp: "Formatted at 2026-09-27 07:01:35 on dist-test-slave-p6hz"
server_key: "9f29986baea22b9dff18d590d07cc052"
server_key_iv: "f2d55be595c2becf215c425034594ef2"
server_key_version: "encryptionkey@0"
I20260927 07:01:35.024161 23554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260927 07:01:35.027716 23568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:35.029175 23554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260927 07:01:35.029353 23554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "b71ce1c68f184b739625b05a6953f87e"
format_stamp: "Formatted at 2026-09-27 07:01:35 on dist-test-slave-p6hz"
server_key: "9f29986baea22b9dff18d590d07cc052"
server_key_iv: "f2d55be595c2becf215c425034594ef2"
server_key_version: "encryptionkey@0"
I20260927 07:01:35.029569 23554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:35.035945 23554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:35.036764 23554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:35.037051 23554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:35.054656 23554 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:39783
I20260927 07:01:35.054736 23619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:39783 every 8 connection(s)
I20260927 07:01:35.056139 23554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:01:35.060094 23620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:35.062273 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 23554
I20260927 07:01:35.062487 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:01:35.063293 13799 external_mini_cluster.cc:1428] Setting key b503b241848801b7d532ffbafa56ea78
I20260927 07:01:35.072008 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:35.065580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38018 (local address 127.13.121.254:39783)
0927 07:01:35.066665 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:35.066678 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:35.066720 (+ 42us) server_negotiation.cc:407] Connection header received
0927 07:01:35.068391 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:35.068416 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:35.068853 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:35.069220 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:35.070310 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:35.070316 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:35.070330 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:35.070345 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:35.070416 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:35.070432 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:35.070673 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:35.070807 (+ 134us) server_negotiation.cc:299] Negotiation successful
0927 07:01:35.071049 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":369,"threads_started":1}
I20260927 07:01:35.072700 23620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e: Bootstrap starting.
I20260927 07:01:35.075791 23620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:35.076750 23620 log.cc:824] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:35.079636 23620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e: No bootstrap required, opened a new log
I20260927 07:01:35.086941 23620 raft_consensus.cc:348] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71ce1c68f184b739625b05a6953f87e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39783 } }
I20260927 07:01:35.087373 23620 raft_consensus.cc:374] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:35.087476 23620 raft_consensus.cc:729] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b71ce1c68f184b739625b05a6953f87e, State: Initialized, Role: FOLLOWER
I20260927 07:01:35.088037 23620 consensus_queue.cc:260] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71ce1c68f184b739625b05a6953f87e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39783 } }
I20260927 07:01:35.088256 23620 raft_consensus.cc:388] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:01:35.088382 23620 raft_consensus.cc:482] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:01:35.088560 23620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:35.090647 23620 raft_consensus.cc:504] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71ce1c68f184b739625b05a6953f87e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39783 } }
I20260927 07:01:35.091171 23620 leader_election.cc:302] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b71ce1c68f184b739625b05a6953f87e; no voters:
I20260927 07:01:35.091897 23620 leader_election.cc:288] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:01:35.092018 23625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:35.092773 23625 raft_consensus.cc:686] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [term 1 LEADER]: Becoming Leader. State: Replica: b71ce1c68f184b739625b05a6953f87e, State: Running, Role: LEADER
I20260927 07:01:35.093770 23625 consensus_queue.cc:237] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71ce1c68f184b739625b05a6953f87e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39783 } }
I20260927 07:01:35.094203 23620 sys_catalog.cc:564] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:01:35.096356 23627 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b71ce1c68f184b739625b05a6953f87e. Latest consensus state: current_term: 1 leader_uuid: "b71ce1c68f184b739625b05a6953f87e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71ce1c68f184b739625b05a6953f87e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39783 } } }
I20260927 07:01:35.096717 23627 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:35.097250 23626 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b71ce1c68f184b739625b05a6953f87e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71ce1c68f184b739625b05a6953f87e" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 39783 } } }
I20260927 07:01:35.097594 23626 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:35.097631 23631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:01:35.103747 23631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:01:35.115352 23631 catalog_manager.cc:1347] Generated new cluster ID: 98834fb2290244eb82423241afe376c6
I20260927 07:01:35.115463 23631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:01:35.129365 23631 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:01:35.129945 23631 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:01:35.144985 23631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b71ce1c68f184b739625b05a6953f87e: Generated new TSK 0
I20260927 07:01:35.145830 23631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:01:35.155694 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39783
--builtin_ntp_servers=127.13.121.212:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 07:01:35.498600 23645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:35.498893 23645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:35.499018 23645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:35.506459 23645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:35.506582 23645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:35.506721 23645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:01:35.516052 23645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:35.517319 23645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:35.519080 23645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:35.532423 23650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:35.532387 23653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:35.532459 23651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:35.533497 23645 server_base.cc:1034] running on GCE node
I20260927 07:01:35.534197 23645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:35.535362 23645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:35.536610 23645 hybrid_clock.cc:648] HybridClock initialized: now 1790492495536544 us; error 55 us; skew 500 ppm
I20260927 07:01:35.536970 23645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:01:35.539564 23645 webserver.cc:460] Webserver started at http://127.13.121.193:44015/ using document root <none> and password file <none>
I20260927 07:01:35.540329 23645 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:35.540460 23645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:35.540781 23645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:35.545037 23645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "3b0f595d280d42b4af1c858545da3a80"
format_stamp: "Formatted at 2026-09-27 07:01:35 on dist-test-slave-p6hz"
server_key: "497fdea0dcac6bb783d26be3b475197a"
server_key_iv: "0d4224463ad446510507da94ec432756"
server_key_version: "encryptionkey@0"
I20260927 07:01:35.545840 23645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "3b0f595d280d42b4af1c858545da3a80"
format_stamp: "Formatted at 2026-09-27 07:01:35 on dist-test-slave-p6hz"
server_key: "497fdea0dcac6bb783d26be3b475197a"
server_key_iv: "0d4224463ad446510507da94ec432756"
server_key_version: "encryptionkey@0"
I20260927 07:01:35.550879 23645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260927 07:01:35.554191 23659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:35.555521 23645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:01:35.555666 23645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "3b0f595d280d42b4af1c858545da3a80"
format_stamp: "Formatted at 2026-09-27 07:01:35 on dist-test-slave-p6hz"
server_key: "497fdea0dcac6bb783d26be3b475197a"
server_key_iv: "0d4224463ad446510507da94ec432756"
server_key_version: "encryptionkey@0"
I20260927 07:01:35.555892 23645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:35.561430 23645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:35.562230 23645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:35.562484 23645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:35.563436 23645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:35.564855 23645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:35.564949 23645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:35.565061 23645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:35.565140 23645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:35.594655 23645 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:34619
I20260927 07:01:35.594697 23771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:34619 every 8 connection(s)
I20260927 07:01:35.596028 23645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:01:35.603659 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 23645
I20260927 07:01:35.603869 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:01:35.604012 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:35.598433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:36449 (local address 127.13.121.254:39783)
0927 07:01:35.598844 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:35.598849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:35.600156 (+ 1307us) server_negotiation.cc:407] Connection header received
0927 07:01:35.601763 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:35.601769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:35.601834 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:35.601928 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:35.603197 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:35.603203 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:35.603208 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:35.603228 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:35.603278 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:35.603293 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:35.603449 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:35.603674 (+ 225us) server_negotiation.cc:299] Negotiation successful
0927 07:01:35.603736 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":203,"threads_started":1}
I20260927 07:01:35.604578 13799 external_mini_cluster.cc:1428] Setting key 6355f48af686419da9f841c99e5f3350
I20260927 07:01:35.605019 23775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:35.599013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39783 (local address 127.13.121.193:36449)
0927 07:01:35.599997 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:35.600038 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:35.601494 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:35.602159 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:35.602168 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:35.602256 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:35.602861 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:01:35.602864 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:35.602900 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:01:35.602999 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:35.603425 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:35.603437 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:35.603446 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 07:01:35.603785 (+ 339us) client_negotiation.cc:240] Negotiation successful
0927 07:01:35.604036 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":333,"threads_started":1}
I20260927 07:01:35.606539 23772 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39783
I20260927 07:01:35.606853 23772 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:35.607548 23772 heartbeater.cc:511] Master 127.13.121.254:39783 requested a full tablet report, sending...
I20260927 07:01:35.607542 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39783
--builtin_ntp_servers=127.13.121.212:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260927 07:01:35.609580 23585 ts_manager.cc:194] Registered new tserver with Master: 3b0f595d280d42b4af1c858545da3a80 (127.13.121.193:34619)
W20260927 07:01:35.928138 23776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:35.928442 23776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:35.928560 23776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:35.935973 23776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:35.936103 23776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:35.936237 23776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:01:35.945442 23776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:35.946719 23776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:35.948611 23776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:35.961982 23782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:35.962014 23781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:35.961948 23784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:35.963124 23776 server_base.cc:1034] running on GCE node
I20260927 07:01:35.963773 23776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:35.964748 23776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:35.966033 23776 hybrid_clock.cc:648] HybridClock initialized: now 1790492495965958 us; error 68 us; skew 500 ppm
I20260927 07:01:35.966339 23776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:01:35.968583 23776 webserver.cc:460] Webserver started at http://127.13.121.194:33497/ using document root <none> and password file <none>
I20260927 07:01:35.969249 23776 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:35.969345 23776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:35.969645 23776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:35.972450 23776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "52c5cef844244d3591eaeb43a703e50c"
format_stamp: "Formatted at 2026-09-27 07:01:35 on dist-test-slave-p6hz"
server_key: "3af2679ed4b63e0244f9e094302511aa"
server_key_iv: "3dc382bbbece72bee58d1dd4f67e7857"
server_key_version: "encryptionkey@0"
I20260927 07:01:35.973217 23776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "52c5cef844244d3591eaeb43a703e50c"
format_stamp: "Formatted at 2026-09-27 07:01:35 on dist-test-slave-p6hz"
server_key: "3af2679ed4b63e0244f9e094302511aa"
server_key_iv: "3dc382bbbece72bee58d1dd4f67e7857"
server_key_version: "encryptionkey@0"
I20260927 07:01:35.978190 23776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260927 07:01:35.981621 23790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:35.982977 23776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:01:35.983166 23776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "52c5cef844244d3591eaeb43a703e50c"
format_stamp: "Formatted at 2026-09-27 07:01:35 on dist-test-slave-p6hz"
server_key: "3af2679ed4b63e0244f9e094302511aa"
server_key_iv: "3dc382bbbece72bee58d1dd4f67e7857"
server_key_version: "encryptionkey@0"
I20260927 07:01:35.983385 23776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:35.988879 23776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:35.989622 23776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:35.989856 23776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:35.990744 23776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:35.992165 23776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:35.992256 23776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:35.992362 23776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:35.992429 23776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:36.021747 23776 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:34857
I20260927 07:01:36.021798 23902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:34857 every 8 connection(s)
I20260927 07:01:36.023029 23776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:01:36.028508 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 23776
I20260927 07:01:36.028707 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:01:36.029102 13799 external_mini_cluster.cc:1428] Setting key 10d84db4fe9c14286ed3cabe1a0f3b80
I20260927 07:01:36.031181 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:36.025717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:55691 (local address 127.13.121.254:39783)
0927 07:01:36.025894 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:36.025899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:36.027179 (+ 1280us) server_negotiation.cc:407] Connection header received
0927 07:01:36.028860 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:36.028866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:36.028937 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:36.029022 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:36.030407 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:36.030412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:36.030415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:36.030432 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:36.030462 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:36.030474 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:36.030589 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:36.030818 (+ 229us) server_negotiation.cc:299] Negotiation successful
0927 07:01:36.030884 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 07:01:36.032321 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39783
--builtin_ntp_servers=127.13.121.212:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260927 07:01:36.032580 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:36.026150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39783 (local address 127.13.121.194:55691)
0927 07:01:36.026986 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:36.027024 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:36.028580 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:36.029293 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:36.029303 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:36.029400 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:36.030066 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:01:36.030069 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:36.030122 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:01:36.030236 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:36.030571 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:36.030581 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:36.030589 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:01:36.030867 (+ 278us) client_negotiation.cc:240] Negotiation successful
0927 07:01:36.031205 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":245,"threads_started":1}
I20260927 07:01:36.034019 23903 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39783
I20260927 07:01:36.034356 23903 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:36.035032 23903 heartbeater.cc:511] Master 127.13.121.254:39783 requested a full tablet report, sending...
I20260927 07:01:36.036567 23585 ts_manager.cc:194] Registered new tserver with Master: 52c5cef844244d3591eaeb43a703e50c (127.13.121.194:34857)
W20260927 07:01:36.369405 23906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:36.369712 23906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:36.369823 23906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:36.377351 23906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:36.377485 23906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:36.377717 23906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:01:36.387512 23906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:39783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:36.388787 23906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:36.390545 23906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:36.403405 23911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:36.403362 23912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:36.403328 23914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:36.404639 23906 server_base.cc:1034] running on GCE node
I20260927 07:01:36.405197 23906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:36.406188 23906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:36.407404 23906 hybrid_clock.cc:648] HybridClock initialized: now 1790492496407347 us; error 43 us; skew 500 ppm
I20260927 07:01:36.407835 23906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:01:36.410212 23906 webserver.cc:460] Webserver started at http://127.13.121.195:39843/ using document root <none> and password file <none>
I20260927 07:01:36.410921 23906 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:36.411027 23906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:36.411386 23906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:36.414302 23906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "ac39bcaef9dd4a1d8a741905373f7078"
format_stamp: "Formatted at 2026-09-27 07:01:36 on dist-test-slave-p6hz"
server_key: "dcdd511d22b6f72ebf48044d64dfe120"
server_key_iv: "c1801d0c9186e006ee8afc2726995a71"
server_key_version: "encryptionkey@0"
I20260927 07:01:36.415081 23906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "ac39bcaef9dd4a1d8a741905373f7078"
format_stamp: "Formatted at 2026-09-27 07:01:36 on dist-test-slave-p6hz"
server_key: "dcdd511d22b6f72ebf48044d64dfe120"
server_key_iv: "c1801d0c9186e006ee8afc2726995a71"
server_key_version: "encryptionkey@0"
I20260927 07:01:36.420559 23906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260927 07:01:36.424253 23920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:36.425805 23906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:01:36.425967 23906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "ac39bcaef9dd4a1d8a741905373f7078"
format_stamp: "Formatted at 2026-09-27 07:01:36 on dist-test-slave-p6hz"
server_key: "dcdd511d22b6f72ebf48044d64dfe120"
server_key_iv: "c1801d0c9186e006ee8afc2726995a71"
server_key_version: "encryptionkey@0"
I20260927 07:01:36.426201 23906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:36.432288 23906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:36.433171 23906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:36.433425 23906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:36.434394 23906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:36.435913 23906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:36.436000 23906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:36.436108 23906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:36.436173 23906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:36.467013 23906 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:43287
I20260927 07:01:36.467058 24032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:43287 every 8 connection(s)
I20260927 07:01:36.468379 23906 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:01:36.475626 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 23906
I20260927 07:01:36.475826 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:01:36.476326 13799 external_mini_cluster.cc:1428] Setting key f6f77b37089cdd0495622e674ef5cb0a
I20260927 07:01:36.476678 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:36.470908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:55529 (local address 127.13.121.254:39783)
0927 07:01:36.471191 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:36.471197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:36.472569 (+ 1372us) server_negotiation.cc:407] Connection header received
0927 07:01:36.474232 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:36.474237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:36.474304 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:36.474389 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:36.475946 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:36.475952 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:36.475955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:36.475975 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:36.476008 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:36.476021 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:36.476166 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:36.476380 (+ 214us) server_negotiation.cc:299] Negotiation successful
0927 07:01:36.476447 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260927 07:01:36.477893 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:36.471486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:39783 (local address 127.13.121.195:55529)
0927 07:01:36.472404 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:36.472445 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:36.473937 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:36.474644 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:36.474654 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:36.474757 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:36.475500 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:01:36.475503 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:36.475574 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:01:36.475695 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:36.476157 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:36.476166 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:36.476174 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:01:36.476454 (+ 280us) client_negotiation.cc:240] Negotiation successful
0927 07:01:36.476746 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":261,"threads_started":1}
I20260927 07:01:36.479214 24033 heartbeater.cc:347] Connected to a master server at 127.13.121.254:39783
I20260927 07:01:36.479523 24033 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:36.480144 24033 heartbeater.cc:511] Master 127.13.121.254:39783 requested a full tablet report, sending...
I20260927 07:01:36.481447 23585 ts_manager.cc:194] Registered new tserver with Master: ac39bcaef9dd4a1d8a741905373f7078 (127.13.121.195:43287)
I20260927 07:01:36.492050 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 07:01:36.498005 23774 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:01:36.496463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.13.121.254:39783)
0927 07:01:36.496674 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:36.496681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:36.497058 (+ 377us) server_negotiation.cc:407] Connection header received
0927 07:01:36.497269 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:36.497273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:36.497328 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:36.497413 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:36.497752 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38024: BlockingRecv error: recv got EOF from 127.0.0.1:38024 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 07:01:36.498895 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 23645
I20260927 07:01:36.502143 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:36.499869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.13.121.193:34619)
0927 07:01:36.500346 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:36.500353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:36.500378 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:01:36.500431 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:36.500436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:36.500645 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:36.500762 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:36.501510 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:36.501516 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:36.501527 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:36.501544 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:36.501583 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:36.501597 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:36.501694 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:36.501764 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 07:01:36.501844 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":258,"threads_started":1}
I20260927 07:01:36.503335 23767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:36.583225 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 23645
I20260927 07:01:36.596982 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 23776
I20260927 07:01:36.600241 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:36.597914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.13.121.194:34857)
0927 07:01:36.598492 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:36.598499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:36.598526 (+ 27us) server_negotiation.cc:407] Connection header received
0927 07:01:36.598585 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:36.598591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:36.598807 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:36.598933 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:36.599694 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:36.599701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:36.599712 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:36.599728 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:36.599763 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:36.599777 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:36.599878 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:36.599929 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 07:01:36.600002 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":318,"threads_started":1}
I20260927 07:01:36.601435 23898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:36.681559 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 23776
I20260927 07:01:36.694892 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 23906
I20260927 07:01:36.698686 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:36.695790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.13.121.195:43287)
0927 07:01:36.696456 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:36.696463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:36.696496 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:01:36.696559 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:36.696566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:36.696782 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:36.696912 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:36.697820 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:36.697826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:36.697838 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:01:36.697851 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:36.697895 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:01:36.697912 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:36.698056 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:36.698279 (+ 223us) server_negotiation.cc:299] Negotiation successful
0927 07:01:36.698385 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":340,"threads_started":1}
I20260927 07:01:36.700039 24028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:36.781816 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 23906
I20260927 07:01:36.795341 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 23554
I20260927 07:01:36.796533 23615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:36.895977 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 23554
2026-09-27T07:01:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2312 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:01:37 dist-test-slave-p6hz krb5kdc[24051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:01:37 dist-test-slave-p6hz krb5kdc[24051](info): set up 2 sockets
Sep 27 07:01:37 dist-test-slave-p6hz krb5kdc[24051](info): commencing operation
krb5kdc: starting...
W20260927 07:01:37.919961 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:01:38 dist-test-slave-p6hz krb5kdc[24051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:01:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:01:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:01:38.151964 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:43527
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:43527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:38.473816 24064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:38.474154 24064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:38.474280 24064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:38.481539 24064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:01:38.481675 24064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:38.481742 24064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:38.481819 24064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:01:38.481880 24064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:01:38.491918 24064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:43527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:43527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:38.493280 24064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:38.494589 24064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:38.505888 24069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:38.505962 24070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:38.505986 24072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:38.506752 24064 server_base.cc:1034] running on GCE node
I20260927 07:01:38.507599 24064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:38.509051 24064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:38.510308 24064 hybrid_clock.cc:648] HybridClock initialized: now 1790492498510256 us; error 38 us; skew 500 ppm
I20260927 07:01:38.510676 24064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:38 dist-test-slave-p6hz krb5kdc[24051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492498, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:38.513792 24064 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:01:38.515440 24064 webserver.cc:460] Webserver started at http://127.13.121.254:46159/ using document root <none> and password file <none>
I20260927 07:01:38.516134 24064 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:38.516247 24064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:38.516552 24064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:38.519680 24064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "6d0a25b4f7ce4e51be84641a393c36af"
format_stamp: "Formatted at 2026-09-27 07:01:38 on dist-test-slave-p6hz"
server_key: "2903d7e00178f8615040d8b7a782db0f"
server_key_iv: "d97dd716dcc28fd0eb58d45f46b049c8"
server_key_version: "encryptionkey@0"
I20260927 07:01:38.520479 24064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "6d0a25b4f7ce4e51be84641a393c36af"
format_stamp: "Formatted at 2026-09-27 07:01:38 on dist-test-slave-p6hz"
server_key: "2903d7e00178f8615040d8b7a782db0f"
server_key_iv: "d97dd716dcc28fd0eb58d45f46b049c8"
server_key_version: "encryptionkey@0"
I20260927 07:01:38.526050 24064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:01:38.530026 24079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:38.531668 24064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260927 07:01:38.531859 24064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "6d0a25b4f7ce4e51be84641a393c36af"
format_stamp: "Formatted at 2026-09-27 07:01:38 on dist-test-slave-p6hz"
server_key: "2903d7e00178f8615040d8b7a782db0f"
server_key_iv: "d97dd716dcc28fd0eb58d45f46b049c8"
server_key_version: "encryptionkey@0"
I20260927 07:01:38.532060 24064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:38.554626 24064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:38.559034 24064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:38.559350 24064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:38.577340 24064 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:43527
I20260927 07:01:38.577361 24130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:43527 every 8 connection(s)
I20260927 07:01:38.578698 24064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:01:38.582428 24131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:38.584338 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 24064
I20260927 07:01:38.584540 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:01:38.585111 13799 external_mini_cluster.cc:1428] Setting key 0329fdca2b52d24b7a6af29d8da8f125
I20260927 07:01:38.595449 24131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af: Bootstrap starting.
I20260927 07:01:38.598874 24131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:38.599922 24131 log.cc:824] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af: Log is configured to *not* fsync() on all Append() calls
Sep 27 07:01:38 dist-test-slave-p6hz krb5kdc[24051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:01:38.603467 24131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af: No bootstrap required, opened a new log
I20260927 07:01:38.609638 24134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:38.587185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34200 (local address 127.13.121.254:43527)
0927 07:01:38.588004 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:38.588019 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:38.588078 (+ 59us) server_negotiation.cc:407] Connection header received
0927 07:01:38.590306 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:38.590337 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:38.590852 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:38.591335 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:38.592634 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:38.594650 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:38.595989 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:38.596969 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:38.597191 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:38.601180 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:38.601189 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:38.601204 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:38.601242 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:38.605049 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:38.605777 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:38.605787 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:38.605795 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:38.606015 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:38.606378 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:38.606383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:38.606387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:38.607118 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:38.607449 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:38.607748 (+ 299us) server_negotiation.cc:299] Negotiation successful
0927 07:01:38.608240 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":331,"threads_started":1}
I20260927 07:01:38.613500 24131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a25b4f7ce4e51be84641a393c36af" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43527 } }
I20260927 07:01:38.613858 24131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:38.613932 24131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0a25b4f7ce4e51be84641a393c36af, State: Initialized, Role: FOLLOWER
I20260927 07:01:38.614472 24131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a25b4f7ce4e51be84641a393c36af" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43527 } }
I20260927 07:01:38.614710 24131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:01:38.614852 24131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:01:38.615046 24131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:38.617290 24131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a25b4f7ce4e51be84641a393c36af" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43527 } }
I20260927 07:01:38.617830 24131 leader_election.cc:302] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d0a25b4f7ce4e51be84641a393c36af; no voters:
I20260927 07:01:38.618494 24131 leader_election.cc:288] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:01:38.618644 24136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:38.619378 24136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [term 1 LEADER]: Becoming Leader. State: Replica: 6d0a25b4f7ce4e51be84641a393c36af, State: Running, Role: LEADER
I20260927 07:01:38.619961 24136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a25b4f7ce4e51be84641a393c36af" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43527 } }
I20260927 07:01:38.620802 24131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:01:38.623262 24137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d0a25b4f7ce4e51be84641a393c36af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a25b4f7ce4e51be84641a393c36af" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43527 } } }
I20260927 07:01:38.623054 24138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d0a25b4f7ce4e51be84641a393c36af. Latest consensus state: current_term: 1 leader_uuid: "6d0a25b4f7ce4e51be84641a393c36af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0a25b4f7ce4e51be84641a393c36af" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 43527 } } }
I20260927 07:01:38.623598 24137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:38.623598 24138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af [sys.catalog]: This master's current role is: LEADER
I20260927 07:01:38.624434 24144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:01:38.629415 24144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:01:38.638404 24144 catalog_manager.cc:1347] Generated new cluster ID: 40de251f3ee74762aff0c28caac870fb
I20260927 07:01:38.638530 24144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:01:38.652185 24144 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:01:38.654062 24144 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:01:38.667780 24144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af: Generated new TSK 0
I20260927 07:01:38.668932 24144 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:01:38.732300 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43527
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:39.061350 24157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:39.061667 24157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:39.061794 24157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:39.069147 24157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:39.069294 24157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:39.069445 24157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:01:39.079373 24157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:39.080705 24157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:39.082118 24157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:39.095343 24162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:39.095325 24163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:39.095616 24165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:39.096251 24157 server_base.cc:1034] running on GCE node
I20260927 07:01:39.096836 24157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:39.097748 24157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:39.098974 24157 hybrid_clock.cc:648] HybridClock initialized: now 1790492499098923 us; error 34 us; skew 500 ppm
I20260927 07:01:39.099380 24157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:39 dist-test-slave-p6hz krb5kdc[24051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:39.102824 24157 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:01:39.104513 24157 webserver.cc:460] Webserver started at http://127.13.121.193:41047/ using document root <none> and password file <none>
I20260927 07:01:39.105217 24157 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:39.105322 24157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:39.105592 24157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:39.108542 24157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "9df70546b1b746cfb010eb8aa8afb021"
format_stamp: "Formatted at 2026-09-27 07:01:39 on dist-test-slave-p6hz"
server_key: "2a2e8daac0e7c4a0d8995a2588d1ab19"
server_key_iv: "5aea463feae8fa325b89ccfe46db1e3b"
server_key_version: "encryptionkey@0"
I20260927 07:01:39.109475 24157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "9df70546b1b746cfb010eb8aa8afb021"
format_stamp: "Formatted at 2026-09-27 07:01:39 on dist-test-slave-p6hz"
server_key: "2a2e8daac0e7c4a0d8995a2588d1ab19"
server_key_iv: "5aea463feae8fa325b89ccfe46db1e3b"
server_key_version: "encryptionkey@0"
I20260927 07:01:39.116997 24157 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260927 07:01:39.120932 24172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:39.122471 24157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:01:39.122669 24157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "9df70546b1b746cfb010eb8aa8afb021"
format_stamp: "Formatted at 2026-09-27 07:01:39 on dist-test-slave-p6hz"
server_key: "2a2e8daac0e7c4a0d8995a2588d1ab19"
server_key_iv: "5aea463feae8fa325b89ccfe46db1e3b"
server_key_version: "encryptionkey@0"
I20260927 07:01:39.122880 24157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:39.137888 24157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:39.142535 24157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:39.142899 24157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:39.144254 24157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:39.145639 24157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:39.145711 24157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:39.145792 24157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:39.145836 24157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:39.175369 24157 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:38409
I20260927 07:01:39.175405 24284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:38409 every 8 connection(s)
I20260927 07:01:39.176579 24157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:01:39.177104 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 24157
I20260927 07:01:39.177326 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:01:39.178063 13799 external_mini_cluster.cc:1428] Setting key 0004a780eacdee8af2b3700fa2fb8133
Sep 27 07:01:39 dist-test-slave-p6hz krb5kdc[24051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:01:39.197393 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:39.179222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:44465 (local address 127.13.121.254:43527)
0927 07:01:39.180138 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:39.180145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:39.180715 (+ 570us) server_negotiation.cc:407] Connection header received
0927 07:01:39.183184 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:39.183190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:39.183269 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:39.183358 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:39.185263 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:39.185955 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:39.188104 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:39.188495 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:39.188614 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:39.191934 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:39.191939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:39.191942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:39.191973 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:39.194402 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:39.195131 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:39.195168 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:39.195171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:39.195291 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:39.195719 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:39.195725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:39.195728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:39.195977 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:39.196127 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:39.196822 (+ 695us) server_negotiation.cc:299] Negotiation successful
0927 07:01:39.197038 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796,"thread_start_us":701,"threads_started":1}
I20260927 07:01:39.198355 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:39.179680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:43527 (local address 127.13.121.193:44465)
0927 07:01:39.180536 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:39.180591 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:39.182808 (+ 2217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:39.183629 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:39.183641 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:39.184316 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:39.185074 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:39.185101 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:39.186124 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:39.186129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:39.187930 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:39.187946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:39.188731 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:39.188739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:39.188957 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:39.189606 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:39.189627 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:39.191710 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:39.194562 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:39.194571 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:39.194585 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:39.194997 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:39.195396 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:39.195401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:39.195405 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:39.195604 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:39.196110 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:39.196118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:39.196453 (+ 335us) client_negotiation.cc:769] Sending connection context
0927 07:01:39.196689 (+ 236us) client_negotiation.cc:240] Negotiation successful
0927 07:01:39.197217 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":396,"threads_started":1}
I20260927 07:01:39.199945 24285 heartbeater.cc:347] Connected to a master server at 127.13.121.254:43527
I20260927 07:01:39.200268 24285 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:01:39.200953 24285 heartbeater.cc:511] Master 127.13.121.254:43527 requested a full tablet report, sending...
I20260927 07:01:39.202996 24096 ts_manager.cc:194] Registered new tserver with Master: 9df70546b1b746cfb010eb8aa8afb021 (127.13.121.193:38409)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
I20260927 07:01:39.204535 24096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:44465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:01:39.231652 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43527
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:39.557370 24291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:39.557670 24291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:39.557762 24291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:39.565212 24291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:39.565335 24291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:39.565450 24291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:01:39.575224 24291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:39.576478 24291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:39.578157 24291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:39.592700 24296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:39.592784 24297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:39.592784 24299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:39.593515 24291 server_base.cc:1034] running on GCE node
I20260927 07:01:39.594178 24291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:39.595188 24291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:39.596405 24291 hybrid_clock.cc:648] HybridClock initialized: now 1790492499596354 us; error 41 us; skew 500 ppm
I20260927 07:01:39.596722 24291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:39 dist-test-slave-p6hz krb5kdc[24051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:39.599674 24291 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:01:39.601078 24291 webserver.cc:460] Webserver started at http://127.13.121.194:33177/ using document root <none> and password file <none>
I20260927 07:01:39.601891 24291 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:39.602008 24291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:39.602288 24291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:39.605198 24291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f"
format_stamp: "Formatted at 2026-09-27 07:01:39 on dist-test-slave-p6hz"
server_key: "8b592a1e867d7ce74cc46a34c19a0534"
server_key_iv: "4b0ebc6197148fdd58a4b41fe6c6d934"
server_key_version: "encryptionkey@0"
I20260927 07:01:39.605966 24291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f"
format_stamp: "Formatted at 2026-09-27 07:01:39 on dist-test-slave-p6hz"
server_key: "8b592a1e867d7ce74cc46a34c19a0534"
server_key_iv: "4b0ebc6197148fdd58a4b41fe6c6d934"
server_key_version: "encryptionkey@0"
I20260927 07:01:39.611115 24291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260927 07:01:39.614571 24306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:39.616070 24291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 07:01:39.616220 24291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f"
format_stamp: "Formatted at 2026-09-27 07:01:39 on dist-test-slave-p6hz"
server_key: "8b592a1e867d7ce74cc46a34c19a0534"
server_key_iv: "4b0ebc6197148fdd58a4b41fe6c6d934"
server_key_version: "encryptionkey@0"
I20260927 07:01:39.616402 24291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:39.634644 24291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:39.638957 24291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:39.639273 24291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:39.640269 24291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:39.641582 24291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:39.641660 24291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:39.641767 24291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:39.641814 24291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:39.671795 24291 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:38821
I20260927 07:01:39.671844 24418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:38821 every 8 connection(s)
I20260927 07:01:39.673043 24291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:01:39.683367 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 24291
I20260927 07:01:39.683573 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:01:39.683969 13799 external_mini_cluster.cc:1428] Setting key a1730034ac5756cd66ee401eebb02f1e
Sep 27 07:01:39 dist-test-slave-p6hz krb5kdc[24051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:01:39.692061 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:39.675513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:53493 (local address 127.13.121.254:43527)
0927 07:01:39.675682 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:39.675686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:39.676773 (+ 1087us) server_negotiation.cc:407] Connection header received
0927 07:01:39.678413 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:39.678418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:39.678500 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:39.678599 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:39.680018 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:39.680726 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:39.682256 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:39.682752 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:39.682849 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:39.686184 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:39.686190 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:39.686194 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:39.686230 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:39.689117 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:39.689809 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:39.689813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:39.689816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:39.689954 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:39.690369 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:39.690373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:39.690376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:39.690680 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:39.690809 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:39.691528 (+ 719us) server_negotiation.cc:299] Negotiation successful
0927 07:01:39.691732 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 07:01:39.692900 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:39.675948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:43527 (local address 127.13.121.194:53493)
0927 07:01:39.676622 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:39.676658 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:39.678178 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:39.678773 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:39.678781 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:39.679319 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:39.679853 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:39.679869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:39.680892 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:39.680897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:39.682113 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:39.682125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:39.682947 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:39.682952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:39.683225 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:39.683859 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:39.683885 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:39.685951 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:39.689265 (+ 3314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:39.689271 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:39.689291 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:39.689701 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:39.690058 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:39.690060 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:39.690063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:39.690265 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:39.690798 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:39.690803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:39.691184 (+ 381us) client_negotiation.cc:769] Sending connection context
0927 07:01:39.691419 (+ 235us) client_negotiation.cc:240] Negotiation successful
0927 07:01:39.691814 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":247,"threads_started":1}
I20260927 07:01:39.694306 24419 heartbeater.cc:347] Connected to a master server at 127.13.121.254:43527
I20260927 07:01:39.694597 24419 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:39.695369 24419 heartbeater.cc:511] Master 127.13.121.254:43527 requested a full tablet report, sending...
I20260927 07:01:39.696825 24096 ts_manager.cc:194] Registered new tserver with Master: ec5e3b1c98f84aa58bfa236c8d4cb44f (127.13.121.194:38821)
I20260927 07:01:39.697649 24096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:53493
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:01:39.735103 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43527
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:01:40.055265 24424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:01:40.055573 24424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:01:40.055693 24424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:01:40.062898 24424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:01:40.063066 24424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:01:40.063247 24424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:01:40.073105 24424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:43527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:01:40.074415 24424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:01:40.075721 24424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:01:40.088295 24432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:40.088358 24430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:01:40.088327 24429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:01:40.088974 24424 server_base.cc:1034] running on GCE node
I20260927 07:01:40.089614 24424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:01:40.090570 24424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:01:40.091775 24424 hybrid_clock.cc:648] HybridClock initialized: now 1790492500091730 us; error 36 us; skew 500 ppm
I20260927 07:01:40.092130 24424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:01:40 dist-test-slave-p6hz krb5kdc[24051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492500, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:01:40.095260 24424 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:01:40.099696 24424 webserver.cc:460] Webserver started at http://127.13.121.195:43761/ using document root <none> and password file <none>
I20260927 07:01:40.100543 24424 fs_manager.cc:359] Metadata directory not provided
I20260927 07:01:40.100682 24424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:01:40.101006 24424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:01:40.104394 24424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "0fa5c7ad63204b7ab26e2e6de05a375c"
format_stamp: "Formatted at 2026-09-27 07:01:40 on dist-test-slave-p6hz"
server_key: "45efe91cc67011c219644ddc08876382"
server_key_iv: "96d909e6b47845bedef5a8753d92a419"
server_key_version: "encryptionkey@0"
I20260927 07:01:40.105149 24424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "0fa5c7ad63204b7ab26e2e6de05a375c"
format_stamp: "Formatted at 2026-09-27 07:01:40 on dist-test-slave-p6hz"
server_key: "45efe91cc67011c219644ddc08876382"
server_key_iv: "96d909e6b47845bedef5a8753d92a419"
server_key_version: "encryptionkey@0"
I20260927 07:01:40.110409 24424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:01:40.114225 24439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:40.115833 24424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:01:40.115994 24424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "0fa5c7ad63204b7ab26e2e6de05a375c"
format_stamp: "Formatted at 2026-09-27 07:01:40 on dist-test-slave-p6hz"
server_key: "45efe91cc67011c219644ddc08876382"
server_key_iv: "96d909e6b47845bedef5a8753d92a419"
server_key_version: "encryptionkey@0"
I20260927 07:01:40.116204 24424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:01:40.134002 24424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:01:40.138897 24424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:01:40.139384 24424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:01:40.140468 24424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:01:40.141887 24424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:01:40.141963 24424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:40.142067 24424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:01:40.142127 24424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:01:40.172065 24424 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:36127
I20260927 07:01:40.172130 24551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:36127 every 8 connection(s)
I20260927 07:01:40.173481 24424 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:01:40.176999 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 24424
I20260927 07:01:40.177211 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:01:40.177685 13799 external_mini_cluster.cc:1428] Setting key 6fc5c336ec5a3be8334e67f622ad49a8
Sep 27 07:01:40 dist-test-slave-p6hz krb5kdc[24051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492500, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:01:40.194449 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.176197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:34879 (local address 127.13.121.254:43527)
0927 07:01:40.177968 (+ 1771us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.177974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.177999 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:01:40.179938 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.179943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.180024 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.180123 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:40.181758 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.182495 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.184193 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.184646 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.184756 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.188398 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:40.188404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:40.188407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:40.188443 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:40.191064 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.192089 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.192095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.192098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.192280 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.192772 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.192777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.192780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.193157 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:40.193298 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.194014 (+ 716us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.194202 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1635}
I20260927 07:01:40.195443 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.176489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:43527 (local address 127.13.121.195:34879)
0927 07:01:40.177340 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:40.177386 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:40.179640 (+ 2254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:40.180379 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:40.180392 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:40.181008 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:01:40.181572 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.181593 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.182656 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.182660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.184029 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.184043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.184912 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.184918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.185175 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.185835 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:01:40.185861 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:01:40.188144 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:01:40.191268 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:40.191279 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:40.191300 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:40.191926 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:40.192408 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:01:40.192413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:01:40.192417 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:01:40.192642 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:01:40.193281 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:01:40.193287 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:01:40.193639 (+ 352us) client_negotiation.cc:769] Sending connection context
0927 07:01:40.193875 (+ 236us) client_negotiation.cc:240] Negotiation successful
0927 07:01:40.194282 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":336,"threads_started":1}
I20260927 07:01:40.196794 24552 heartbeater.cc:347] Connected to a master server at 127.13.121.254:43527
I20260927 07:01:40.197085 24552 heartbeater.cc:464] Registering TS with master...
I20260927 07:01:40.197772 24552 heartbeater.cc:511] Master 127.13.121.254:43527 requested a full tablet report, sending...
I20260927 07:01:40.199129 24096 ts_manager.cc:194] Registered new tserver with Master: 0fa5c7ad63204b7ab26e2e6de05a375c (127.13.121.195:36127)
I20260927 07:01:40.199960 24096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:34879
I20260927 07:01:40.205113 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:01:40.207566 24285 heartbeater.cc:503] Master 127.13.121.254:43527 was elected leader, sending a full tablet report...
I20260927 07:01:40.221143 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.209388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.13.121.254:43527)
0927 07:01:40.209538 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.209542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.210046 (+ 504us) server_negotiation.cc:407] Connection header received
0927 07:01:40.210149 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.210154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.210220 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.210305 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:40.211700 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.212407 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.213575 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.213992 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.214106 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.216265 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:40.216270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:40.216273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:40.216303 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:40.218547 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.219188 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.219193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.219197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.219317 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.219660 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.219664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.219667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.220042 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:40.220270 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.220623 (+ 353us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.220824 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:01:40.224982 24096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:01:40.232069 24096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:01:40.266984 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.259024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.13.121.193:38409)
0927 07:01:40.259687 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.259696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.259720 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:01:40.259794 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.259799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.259961 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.260152 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:40.261311 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.262616 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.264762 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.265813 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.266064 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.266248 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.266496 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.266666 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":357,"threads_started":1}
I20260927 07:01:40.267212 24568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.258723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:38409 (local address 127.0.0.1:49364)
0927 07:01:40.259314 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:40.259331 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:40.259442 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:40.260705 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:40.260709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:40.260730 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:40.261101 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.261111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.262763 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.262766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.264564 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.264576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.266079 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.266085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.266284 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.266324 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 07:01:40.266431 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 07:01:40.266639 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":1,"thread_start_us":287,"threads_started":1}
I20260927 07:01:40.267377 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.244653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:36127 (local address 127.0.0.1:32812)
0927 07:01:40.258498 (+ 13845us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:40.258514 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:40.258639 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:40.259502 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:40.259509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:40.259540 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:40.259870 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.259879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.262270 (+ 2391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.262275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.264736 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.264750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.266534 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.266540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.266718 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.266748 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 07:01:40.266815 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 07:01:40.266947 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13728,"mutex_wait_us":64,"thread_start_us":12293,"threads_started":1}
I20260927 07:01:40.267872 24565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.244331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.13.121.194:38821)
0927 07:01:40.256685 (+ 12354us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.256697 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.258510 (+ 1813us) server_negotiation.cc:407] Connection header received
0927 07:01:40.259737 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.259744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.259961 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.260152 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:40.260707 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.261936 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.264908 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.266313 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.266441 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.266609 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.267407 (+ 798us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.267597 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12127,"thread_start_us":12375,"threads_started":1}
I20260927 07:01:40.268486 24567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.244985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.13.121.195:36127)
0927 07:01:40.257636 (+ 12651us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.257645 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.258814 (+ 1169us) server_negotiation.cc:407] Connection header received
0927 07:01:40.258928 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.258935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.259245 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.260016 (+ 771us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:40.260521 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.262136 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.264916 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.266313 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.267769 (+ 1456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.267942 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.268006 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.268171 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12466,"thread_start_us":12273,"threads_started":1}
I20260927 07:01:40.271446 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.244152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:38821 (local address 127.0.0.1:46144)
0927 07:01:40.258330 (+ 14178us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:40.258388 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:40.258592 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:40.260154 (+ 1562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:40.260158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:40.260189 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:40.260531 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.260543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.262069 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.262074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.264744 (+ 2670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.264753 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.266800 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.266807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.266959 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.266985 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 07:01:40.268125 (+ 1140us) client_negotiation.cc:240] Negotiation successful
0927 07:01:40.268257 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13012,"mutex_wait_us":14,"thread_start_us":13884,"threads_started":1}
I20260927 07:01:40.271365 24487 tablet_service.cc:1467] Processing CreateTablet for tablet 987080b0d7d94bada66a35f53ac19395 (DEFAULT_TABLE table=test-table [id=d93d475b45a146efbe6bf474eb82373a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:40.271023 24220 tablet_service.cc:1467] Processing CreateTablet for tablet 987080b0d7d94bada66a35f53ac19395 (DEFAULT_TABLE table=test-table [id=d93d475b45a146efbe6bf474eb82373a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:40.272701 24487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987080b0d7d94bada66a35f53ac19395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:40.273797 24220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987080b0d7d94bada66a35f53ac19395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:40.276469 24354 tablet_service.cc:1467] Processing CreateTablet for tablet 987080b0d7d94bada66a35f53ac19395 (DEFAULT_TABLE table=test-table [id=d93d475b45a146efbe6bf474eb82373a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:01:40.278046 24354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987080b0d7d94bada66a35f53ac19395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:01:40.286206 24570 tablet_bootstrap.cc:492] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: Bootstrap starting.
I20260927 07:01:40.289292 24570 tablet_bootstrap.cc:654] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:40.290221 24571 tablet_bootstrap.cc:492] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: Bootstrap starting.
I20260927 07:01:40.290576 24570 log.cc:824] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:40.293761 24573 tablet_bootstrap.cc:492] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: Bootstrap starting.
I20260927 07:01:40.293880 24570 tablet_bootstrap.cc:492] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: No bootstrap required, opened a new log
I20260927 07:01:40.294169 24571 tablet_bootstrap.cc:654] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:40.294180 24570 ts_tablet_manager.cc:1397] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260927 07:01:40.295548 24571 log.cc:824] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:40.297075 24573 tablet_bootstrap.cc:654] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: Neither blocks nor log segments found. Creating new log.
I20260927 07:01:40.298328 24573 log.cc:824] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: Log is configured to *not* fsync() on all Append() calls
I20260927 07:01:40.298789 24571 tablet_bootstrap.cc:492] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: No bootstrap required, opened a new log
I20260927 07:01:40.299100 24571 ts_tablet_manager.cc:1397] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260927 07:01:40.302246 24573 tablet_bootstrap.cc:492] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: No bootstrap required, opened a new log
I20260927 07:01:40.302562 24573 ts_tablet_manager.cc:1397] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260927 07:01:40.302881 24570 raft_consensus.cc:348] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } }
I20260927 07:01:40.303333 24570 raft_consensus.cc:374] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:40.303426 24570 raft_consensus.cc:729] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9df70546b1b746cfb010eb8aa8afb021, State: Initialized, Role: FOLLOWER
I20260927 07:01:40.304004 24570 consensus_queue.cc:260] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } }
I20260927 07:01:40.305548 24570 ts_tablet_manager.cc:1428] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: Time spent starting tablet: real 0.011s user 0.012s sys 0.001s
I20260927 07:01:40.309569 24571 raft_consensus.cc:348] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } }
I20260927 07:01:40.310098 24571 raft_consensus.cc:374] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:40.310204 24571 raft_consensus.cc:729] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa5c7ad63204b7ab26e2e6de05a375c, State: Initialized, Role: FOLLOWER
I20260927 07:01:40.310881 24571 consensus_queue.cc:260] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } }
I20260927 07:01:40.312077 24552 heartbeater.cc:503] Master 127.13.121.254:43527 was elected leader, sending a full tablet report...
I20260927 07:01:40.312855 24571 ts_tablet_manager.cc:1428] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260927 07:01:40.314653 24573 raft_consensus.cc:348] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } }
I20260927 07:01:40.315181 24573 raft_consensus.cc:374] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:01:40.315289 24573 raft_consensus.cc:729] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5e3b1c98f84aa58bfa236c8d4cb44f, State: Initialized, Role: FOLLOWER
I20260927 07:01:40.315865 24573 consensus_queue.cc:260] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } }
I20260927 07:01:40.316854 24419 heartbeater.cc:503] Master 127.13.121.254:43527 was elected leader, sending a full tablet report...
I20260927 07:01:40.317418 24573 ts_tablet_manager.cc:1428] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: Time spent starting tablet: real 0.015s user 0.005s sys 0.008s
I20260927 07:01:40.345363 24577 raft_consensus.cc:482] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:01:40.345644 24577 raft_consensus.cc:504] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } }
I20260927 07:01:40.347254 24577 leader_election.cc:288] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec5e3b1c98f84aa58bfa236c8d4cb44f (127.13.121.194:38821), 9df70546b1b746cfb010eb8aa8afb021 (127.13.121.193:38409)
I20260927 07:01:40.353462 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.347495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:38821 (local address 127.13.121.195:47591)
0927 07:01:40.347696 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:40.347716 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:40.347832 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:40.348300 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:40.348305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:40.348337 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:40.348554 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.348566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.349780 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.349785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.351625 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.351640 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.352927 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.352931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.353061 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.353086 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 07:01:40.353164 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 07:01:40.353257 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":24}
I20260927 07:01:40.353591 24565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.347677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:47591 (local address 127.13.121.194:38821)
0927 07:01:40.347854 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.347859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.347883 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:01:40.347953 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.347958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.348028 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.348127 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:40.348712 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.349637 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.351804 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.352782 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.352932 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.353003 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.353214 (+ 211us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.353345 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 07:01:40.354393 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.348157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:41331 (local address 127.13.121.193:38409)
0927 07:01:40.348343 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.348348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.348618 (+ 270us) server_negotiation.cc:407] Connection header received
0927 07:01:40.348756 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.348761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.348833 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.348924 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:01:40.349388 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.350193 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.352397 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.353513 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.353685 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.353750 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.354135 (+ 385us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.354233 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:01:40.354532 24579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.347888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:38409 (local address 127.13.121.195:41331)
0927 07:01:40.348519 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0927 07:01:40.348537 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:01:40.348650 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:01:40.348935 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:01:40.348938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:01:40.348956 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:01:40.349251 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.349259 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.350358 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.350364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.352228 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:01:40.352240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.353866 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.353871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:01:40.353979 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.354001 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 07:01:40.354076 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 07:01:40.354192 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":1,"thread_start_us":305,"threads_started":1}
I20260927 07:01:40.354614 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987080b0d7d94bada66a35f53ac19395" candidate_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" is_pre_election: true
I20260927 07:01:40.355206 24374 raft_consensus.cc:2454] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa5c7ad63204b7ab26e2e6de05a375c in term 0.
I20260927 07:01:40.355489 24240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987080b0d7d94bada66a35f53ac19395" candidate_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9df70546b1b746cfb010eb8aa8afb021" is_pre_election: true
I20260927 07:01:40.355813 24441 leader_election.cc:302] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa5c7ad63204b7ab26e2e6de05a375c, ec5e3b1c98f84aa58bfa236c8d4cb44f; no voters:
I20260927 07:01:40.355993 24240 raft_consensus.cc:2454] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa5c7ad63204b7ab26e2e6de05a375c in term 0.
I20260927 07:01:40.356185 24577 raft_consensus.cc:2781] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:01:40.356334 24577 raft_consensus.cc:482] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:01:40.356446 24577 raft_consensus.cc:3037] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:40.358639 24577 raft_consensus.cc:504] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } }
I20260927 07:01:40.359272 24577 leader_election.cc:288] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [CANDIDATE]: Term 1 election: Requested vote from peers ec5e3b1c98f84aa58bfa236c8d4cb44f (127.13.121.194:38821), 9df70546b1b746cfb010eb8aa8afb021 (127.13.121.193:38409)
I20260927 07:01:40.359781 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987080b0d7d94bada66a35f53ac19395" candidate_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f"
I20260927 07:01:40.359802 24240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987080b0d7d94bada66a35f53ac19395" candidate_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9df70546b1b746cfb010eb8aa8afb021"
I20260927 07:01:40.359966 24374 raft_consensus.cc:3037] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:40.359987 24240 raft_consensus.cc:3037] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:01:40.362334 24374 raft_consensus.cc:2454] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa5c7ad63204b7ab26e2e6de05a375c in term 1.
I20260927 07:01:40.362346 24240 raft_consensus.cc:2454] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa5c7ad63204b7ab26e2e6de05a375c in term 1.
I20260927 07:01:40.362694 24441 leader_election.cc:302] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa5c7ad63204b7ab26e2e6de05a375c, ec5e3b1c98f84aa58bfa236c8d4cb44f; no voters:
I20260927 07:01:40.363008 24577 raft_consensus.cc:2781] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:01:40.363462 24577 raft_consensus.cc:686] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 1 LEADER]: Becoming Leader. State: Replica: 0fa5c7ad63204b7ab26e2e6de05a375c, State: Running, Role: LEADER
I20260927 07:01:40.364018 24577 consensus_queue.cc:237] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } }
I20260927 07:01:40.368484 24096 catalog_manager.cc:5495] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa5c7ad63204b7ab26e2e6de05a375c (127.13.121.195). New cstate: current_term: 1 leader_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa5c7ad63204b7ab26e2e6de05a375c" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36127 } health_report { overall_health: HEALTHY } } }
I20260927 07:01:40.398921 24567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.393608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32814 (local address 127.13.121.195:36127)
0927 07:01:40.393837 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.393843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.393929 (+ 86us) server_negotiation.cc:407] Connection header received
0927 07:01:40.394159 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.394163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.394240 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.394369 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:01:40.394772 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.395545 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.396840 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.397229 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.397345 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.397822 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:01:40.398344 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:01:40.398482 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.398595 (+ 113us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.398678 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 07:01:40.407116 24240 raft_consensus.cc:1261] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [term 1 FOLLOWER]: Refusing update from remote peer 0fa5c7ad63204b7ab26e2e6de05a375c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:40.407116 24374 raft_consensus.cc:1261] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [term 1 FOLLOWER]: Refusing update from remote peer 0fa5c7ad63204b7ab26e2e6de05a375c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:01:40.408159 24580 consensus_queue.cc:1035] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9df70546b1b746cfb010eb8aa8afb021" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:40.408525 24577 consensus_queue.cc:1035] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec5e3b1c98f84aa58bfa236c8d4cb44f" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:01:40.423023 24585 mvcc.cc:205] Tried to move back new op lower bound from 7333857281657126912 to 7333857281494892544. Current Snapshot: MvccSnapshot[applied={T|T < 7333857281657126912}]
I20260927 07:01:40.423412 24584 mvcc.cc:205] Tried to move back new op lower bound from 7333857281657126912 to 7333857281494892544. Current Snapshot: MvccSnapshot[applied={T|T < 7333857281657126912}]
I20260927 07:01:40.436372 24095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:40.436734 24095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:01:40.440729 24095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d0a25b4f7ce4e51be84641a393c36af: Sending DeleteTablet for 3 replicas of tablet 987080b0d7d94bada66a35f53ac19395
I20260927 07:01:40.442188 24487 tablet_service.cc:1514] Processing DeleteTablet for tablet 987080b0d7d94bada66a35f53ac19395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:40 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:32812
I20260927 07:01:40.442440 24354 tablet_service.cc:1514] Processing DeleteTablet for tablet 987080b0d7d94bada66a35f53ac19395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:40 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:46144
I20260927 07:01:40.442946 24220 tablet_service.cc:1514] Processing DeleteTablet for tablet 987080b0d7d94bada66a35f53ac19395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:01:40 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:49364
I20260927 07:01:40.443827 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 24157
I20260927 07:01:40.454810 24592 tablet_replica.cc:330] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: stopping tablet replica
I20260927 07:01:40.455399 24592 raft_consensus.cc:2229] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:01:40.456107 24592 raft_consensus.cc:2258] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:40.456923 24594 tablet_replica.cc:330] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: stopping tablet replica
I20260927 07:01:40.457370 24594 raft_consensus.cc:2229] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:40.457918 24594 raft_consensus.cc:2258] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:40.460350 24594 ts_tablet_manager.cc:1905] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:40.462283 24593 tablet_replica.cc:330] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: stopping tablet replica
I20260927 07:01:40.462672 24592 ts_tablet_manager.cc:1905] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:40.462723 24593 raft_consensus.cc:2229] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:01:40.463291 24593 raft_consensus.cc:2258] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:01:40.467864 24594 ts_tablet_manager.cc:1918] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:40.468115 24594 log.cc:1196] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/987080b0d7d94bada66a35f53ac19395
Sep 27 07:01:40 dist-test-slave-p6hz krb5kdc[24051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:01:40.468819 24594 ts_tablet_manager.cc:1939] T 987080b0d7d94bada66a35f53ac19395 P 9df70546b1b746cfb010eb8aa8afb021: Deleting consensus metadata
I20260927 07:01:40.470393 24593 ts_tablet_manager.cc:1905] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:01:40.470844 24081 catalog_manager.cc:4841] TS 9df70546b1b746cfb010eb8aa8afb021 (127.13.121.193:38409): tablet 987080b0d7d94bada66a35f53ac19395 (table test-table [id=d93d475b45a146efbe6bf474eb82373a]) successfully deleted
I20260927 07:01:40.471482 24592 ts_tablet_manager.cc:1918] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:40.471657 24592 log.cc:1196] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/987080b0d7d94bada66a35f53ac19395
I20260927 07:01:40.472162 24592 ts_tablet_manager.cc:1939] T 987080b0d7d94bada66a35f53ac19395 P 0fa5c7ad63204b7ab26e2e6de05a375c: Deleting consensus metadata
I20260927 07:01:40.474006 24083 catalog_manager.cc:4841] TS 0fa5c7ad63204b7ab26e2e6de05a375c (127.13.121.195:36127): tablet 987080b0d7d94bada66a35f53ac19395 (table test-table [id=d93d475b45a146efbe6bf474eb82373a]) successfully deleted
I20260927 07:01:40.476502 24593 ts_tablet_manager.cc:1918] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:01:40.476661 24593 log.cc:1196] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/987080b0d7d94bada66a35f53ac19395
I20260927 07:01:40.477108 24593 ts_tablet_manager.cc:1939] T 987080b0d7d94bada66a35f53ac19395 P ec5e3b1c98f84aa58bfa236c8d4cb44f: Deleting consensus metadata
I20260927 07:01:40.478581 24081 catalog_manager.cc:4841] TS ec5e3b1c98f84aa58bfa236c8d4cb44f (127.13.121.194:38821): tablet 987080b0d7d94bada66a35f53ac19395 (table test-table [id=d93d475b45a146efbe6bf474eb82373a]) successfully deleted
I20260927 07:01:40.482071 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.445258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49376 (local address 127.13.121.193:38409)
0927 07:01:40.445445 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.445452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.456139 (+ 10687us) server_negotiation.cc:407] Connection header received
0927 07:01:40.456743 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.456749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.456838 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.456921 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:40.458632 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.459915 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.463667 (+ 3752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.464344 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.464448 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.475041 (+ 10593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:40.475059 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:40.475078 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:40.475125 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:40.479001 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.479857 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.479868 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.479872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.480030 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.480409 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.480414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.480417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.480896 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:40.481195 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.481470 (+ 275us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.481706 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 07:01:40.482821 24280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:40.570935 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 24157
I20260927 07:01:40.586254 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 24291
Sep 27 07:01:40 dist-test-slave-p6hz krb5kdc[24051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:01:40.612335 24565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.586987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.13.121.194:38821)
0927 07:01:40.587517 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.587521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.587537 (+ 16us) server_negotiation.cc:407] Connection header received
0927 07:01:40.587585 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.587589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.587657 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.587741 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:40.589078 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.589787 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.590977 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.591474 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.591585 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.605880 (+ 14295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:40.605897 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:40.605914 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:40.605954 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:40.608838 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.609693 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.609700 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.609703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.609831 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.610200 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.610203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.610205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.610574 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:40.610782 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.611103 (+ 321us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.611353 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422}
I20260927 07:01:40.613297 24414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:40.779623 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 24291
I20260927 07:01:40.795117 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 24424
Sep 27 07:01:40 dist-test-slave-p6hz krb5kdc[24051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:01:40.811049 24567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:01:40.796012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.13.121.195:36127)
0927 07:01:40.796203 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:01:40.796209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:01:40.796229 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:01:40.796400 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:01:40.796406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:01:40.796530 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:01:40.796645 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:01:40.797796 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.798615 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.799889 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:01:40.800344 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:01:40.800464 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:01:40.804882 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:01:40.804897 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:01:40.804914 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:01:40.804955 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:01:40.808248 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.808962 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.808970 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.808972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.809143 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:01:40.809506 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:01:40.809509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:01:40.809511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:01:40.809870 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:01:40.810197 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0927 07:01:40.810472 (+ 275us) server_negotiation.cc:299] Negotiation successful
0927 07:01:40.810685 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:01:40.811806 24547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:40.933256 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 24424
I20260927 07:01:40.949435 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 24064
I20260927 07:01:40.950475 24126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:01:41.055334 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 24064
2026-09-27T07:01:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4160 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:01:41 dist-test-slave-p6hz krb5kdc[24602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:01:41 dist-test-slave-p6hz krb5kdc[24602](info): set up 2 sockets
Sep 27 07:01:41 dist-test-slave-p6hz krb5kdc[24602](info): commencing operation
krb5kdc: starting...
W20260927 07:01:42.075891 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:01:42 dist-test-slave-p6hz krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:01:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.121.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.121.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/rangeradmin_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/rangeradmin_127.13.121.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.121.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.121.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/rangerlookup_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/rangerlookup_127.13.121.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.121.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.121.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/HTTP_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/HTTP_127.13.121.212@KRBTEST.COM.keytab.
I20260927 07:01:42.398468 13799 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task8NDO8F/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/postgres -l logfile start
2026-09-27 07:01:44.984 UTC [24632] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 07:01:44.984 UTC [24632] LOG: listening on IPv4 address "127.13.121.212", port 46261
2026-09-27 07:01:44.988 UTC [24632] LOG: listening on Unix socket "/tmp/.s.PGSQL.46261"
2026-09-27 07:01:45.008 UTC [24635] LOG: database system was shut down at 2026-09-27 07:01:43 UTC
2026-09-27 07:01:45.011 UTC [24632] LOG: database system is ready to accept connections
I20260927 07:01:45.899643 13799 mini_postgres.cc:96] Postgres bound to 46261
2026-09-27 07:01:45.916 UTC [24643] FATAL: database "slave" does not exist
127.13.121.212:46261 - accepting connections
I20260927 07:01:45.916980 13799 mini_ranger.cc:161] Starting Ranger...
I20260927 07:01:45.941856 13799 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 07:01:46.447855 13799 mini_ranger.cc:87] Created ranger Postgres database
I20260927 07:01:46.447989 13799 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-admin
2026-09-27 07:01:46,725 [I] DB FLAVOR :POSTGRES
2026-09-27 07:01:46,725 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 07:01:46,725 [I] Checking connection..
2026-09-27 07:01:46,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 07:01:46,980 [I] Checking connection passed.
2026-09-27 07:01:46,981 [I] --------- Verifying version history table ---------
2026-09-27 07:01:46,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:01:47,223 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 07:01:47,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:01:47,472 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 07:01:47,473 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 07:01:47,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 07:01:47.720 UTC [24707] WARNING: there is no transaction in progress
2026-09-27 07:01:47,726 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 07:01:47,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:01:47,972 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 07:01:47,972 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 07:01:47,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 07:01:48,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 07:01:48,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p6hz.c.gcp-upstream.internal','N') ;"
2026-09-27 07:01:48,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 07:01:48,958 [I] Table x_portal_user does not exist in database ranger
2026-09-27 07:01:48,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 07:01:49,196 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 07:01:49,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 07:01:49,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 07:01:49,679 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 07:01:49,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 07:01:50.706 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.725 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.743 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.759 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.777 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.799 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.858 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.865 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.881 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.892 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.903 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.918 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.930 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.938 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.946 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:50.954 UTC [24815] WARNING: there is no transaction in progress
2026-09-27 07:01:51,733 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 07:01:51,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 07:01:51,973 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 07:01:51,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 07:01:52,221 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 07:01:52,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 07:01:52,478 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 07:01:52,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 07:01:52,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 07:01:52,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p6hz.c.gcp-upstream.internal';"
2026-09-27 07:01:53,225 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 07:01:53,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 07:01:53,483 [I] DB_PATCHES have already been applied
I20260927 07:01:53.489923 13799 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-admin:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task8NDO8F/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 07:01:53.490031 13799 mini_ranger.cc:193] Using host: 127.13.121.212
I20260927 07:01:53.506053 13799 mini_ranger.cc:227] Ranger admin URL: http://127.13.121.212:36687
Sep 27, 2026 7:01:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 7:01:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 7:01:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 7:01:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 7:01:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 7:01:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 7:01:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 07:01:54.550552 13799 mini_ranger.cc:160] Time spent starting Ranger: real 8.634s user 0.002s sys 0.041s
Sep 27, 2026 7:02:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 7:02:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 27, 2026 7:02:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 7:02:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 7:02:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 07:02:17.204324 13799 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.121.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.121.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/rangerkms_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/rangerkms_127.13.121.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.121.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.121.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.121.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/HTTP_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.121.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/HTTP_127.13.121.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 07:02:17 dist-test-slave-p6hz krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492537, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 07:02:17.356424 13799 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 07:02:17.376646 13799 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 07:02:18.335261 13799 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 07:02:18.335392 13799 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-kms
I20260927 07:02:18.335515 13799 mini_ranger_kms.cc:227] Using postgres at 127.13.121.212:46261
2026-09-27 07:02:18,716 [I] DB FLAVOR :POSTGRES
2026-09-27 07:02:18,716 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 07:02:18,717 [I] Checking connection
2026-09-27 07:02:18,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 07:02:18,959 [I] connection success
2026-09-27 07:02:18,960 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 07:02:18,960 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 07:02:18,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 07:02:19,199 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 07:02:19,199 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 07:02:19,199 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 07:02:19,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:46261/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 07:02:19,468 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 07:02:19.936299 13799 mini_ranger_kms.cc:310] Created kms service
I20260927 07:02:20.124135 13799 mini_ranger_kms.cc:326] Created kudu user
I20260927 07:02:20.169221 13799 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 07:02:20 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492537, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:02:20.598008 13799 mini_ranger_kms.cc:384] Added ranger policy
I20260927 07:02:20.598356 13799 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-kms:/tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 07:02:20.598419 13799 mini_ranger_kms.cc:240] Using host: 127.13.121.212
I20260927 07:02:20.615605 13799 mini_ranger_kms.cc:276] Ranger KMS PID: 24996
I20260927 07:02:20.615815 13799 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.121.212:38071
Sep 27, 2026 7:02:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 7:02:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 7:02:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 7:02:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 7:02:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 7:02:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 07:02:22.739046 13799 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.383s user 0.124s sys 0.277s
I20260927 07:02:22.739331 13799 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 07:02:22.739530 13799 mini_ranger_kms.cc:401] 127.13.121.212:38071/kms/v1/keys
Sep 27 07:02:22 dist-test-slave-p6hz krb5kdc[24602](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790492542, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.121.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 07:02:23 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492542, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.121.212@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
Sep 27 07:02:23 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492542, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.121.212@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
Sep 27 07:02:25 dist-test-slave-p6hz krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:02:25.994930 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:45123
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:45123
--ranger_config_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38071/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:02:26.323663 25046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:02:26.323963 25046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:02:26.324048 25046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:02:26.331688 25046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:02:26.331830 25046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:02:26.331875 25046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:02:26.331929 25046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:02:26.331974 25046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:02:26.342335 25046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.121.212:38071/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:45123
--ranger_config_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:45123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:02:26.343639 25046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:02:26.344868 25046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:02:26.356600 25051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:02:26.356527 25052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:02:26.356557 25054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:02:26.357005 25046 server_base.cc:1034] running on GCE node
I20260927 07:02:26.357847 25046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:02:26.359354 25046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:02:26.360605 25046 hybrid_clock.cc:648] HybridClock initialized: now 1790492546360563 us; error 35 us; skew 500 ppm
I20260927 07:02:26.360972 25046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:02:26 dist-test-slave-p6hz krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492546, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:02:26.378108 25046 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:02:26.379877 25046 webserver.cc:460] Webserver started at http://127.13.121.254:35075/ using document root <none> and password file <none>
I20260927 07:02:26.380590 25046 fs_manager.cc:359] Metadata directory not provided
I20260927 07:02:26.380712 25046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:02:26.381024 25046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:02:26 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492546, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:02:26.451522 25046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "89f2c67de62049548aa3c849dbe8f6b6"
format_stamp: "Formatted at 2026-09-27 07:02:26 on dist-test-slave-p6hz"
server_key: "89120ffd836db51ad078702c1ad537af"
server_key_iv: "cad71ba684b6661d578d496ee566c98d"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:26.452419 25046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "89f2c67de62049548aa3c849dbe8f6b6"
format_stamp: "Formatted at 2026-09-27 07:02:26 on dist-test-slave-p6hz"
server_key: "89120ffd836db51ad078702c1ad537af"
server_key_iv: "cad71ba684b6661d578d496ee566c98d"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:26.458043 25046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260927 07:02:26.474805 25061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:02:26.476354 25046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:02:26.476527 25046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "89f2c67de62049548aa3c849dbe8f6b6"
format_stamp: "Formatted at 2026-09-27 07:02:26 on dist-test-slave-p6hz"
server_key: "89120ffd836db51ad078702c1ad537af"
server_key_iv: "cad71ba684b6661d578d496ee566c98d"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:26.476751 25046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:02:26.504539 25046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:02:26.508668 25046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:02:26.508986 25046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:02:26.526146 25046 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:45123
I20260927 07:02:26.526173 25112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:45123 every 8 connection(s)
I20260927 07:02:26.527626 25046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:02:26.530933 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 25046
I20260927 07:02:26.531165 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:02:26.531948 13799 external_mini_cluster.cc:1428] Setting key a33825d7a9479f30fa525a0630ff1d85
I20260927 07:02:26.532322 25113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 07:02:26 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:02:26.546798 25113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6: Bootstrap starting.
I20260927 07:02:26.550208 25113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6: Neither blocks nor log segments found. Creating new log.
I20260927 07:02:26.551542 25113 log.cc:824] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6: Log is configured to *not* fsync() on all Append() calls
I20260927 07:02:26.554411 25113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6: No bootstrap required, opened a new log
I20260927 07:02:26.554966 25116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:26.534060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50430 (local address 127.13.121.254:45123)
0927 07:02:26.534811 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:26.534823 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:26.534866 (+ 43us) server_negotiation.cc:407] Connection header received
0927 07:02:26.536981 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:26.537007 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:26.537432 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:26.537779 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:02:26.539405 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:26.540962 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:26.542710 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:26.543505 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:26.543649 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:26.548059 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:02:26.548069 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:02:26.548084 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:02:26.548123 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:02:26.550990 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:26.551708 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:26.551717 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:26.551725 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:26.551885 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:26.552261 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:26.552264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:26.552266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:26.552822 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:02:26.553077 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:26.553544 (+ 467us) server_negotiation.cc:299] Negotiation successful
0927 07:02:26.553929 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":347,"threads_started":1}
I20260927 07:02:26.562618 25113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f2c67de62049548aa3c849dbe8f6b6" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45123 } }
I20260927 07:02:26.563011 25113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:02:26.563107 25113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f2c67de62049548aa3c849dbe8f6b6, State: Initialized, Role: FOLLOWER
I20260927 07:02:26.563870 25113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f2c67de62049548aa3c849dbe8f6b6" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45123 } }
I20260927 07:02:26.564119 25113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:02:26.564291 25113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:02:26.564445 25113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:02:26.566551 25113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f2c67de62049548aa3c849dbe8f6b6" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45123 } }
I20260927 07:02:26.567049 25113 leader_election.cc:302] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89f2c67de62049548aa3c849dbe8f6b6; no voters:
I20260927 07:02:26.567638 25113 leader_election.cc:288] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:02:26.567806 25118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:02:26.568676 25118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [term 1 LEADER]: Becoming Leader. State: Replica: 89f2c67de62049548aa3c849dbe8f6b6, State: Running, Role: LEADER
I20260927 07:02:26.569218 25118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f2c67de62049548aa3c849dbe8f6b6" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45123 } }
I20260927 07:02:26.569653 25113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:02:26.571762 25119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89f2c67de62049548aa3c849dbe8f6b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f2c67de62049548aa3c849dbe8f6b6" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45123 } } }
I20260927 07:02:26.572140 25119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [sys.catalog]: This master's current role is: LEADER
I20260927 07:02:26.573529 25126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:02:26.573592 25120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89f2c67de62049548aa3c849dbe8f6b6. Latest consensus state: current_term: 1 leader_uuid: "89f2c67de62049548aa3c849dbe8f6b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f2c67de62049548aa3c849dbe8f6b6" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45123 } } }
I20260927 07:02:26.573891 25120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6 [sys.catalog]: This master's current role is: LEADER
I20260927 07:02:26.578661 25126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:02:26.584069 25113 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 07:02:26.595279 25126 catalog_manager.cc:1347] Generated new cluster ID: d93c35daa13c46da8ff02fab762ebd0d
I20260927 07:02:26.595539 25126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:02:26.610760 25126 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:02:26.613334 25126 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:02:26.628959 25126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6: Generated new TSK 0
I20260927 07:02:26.630196 25126 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:02:27.419919 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:45123
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:02:27 dist-test-slave-p6hz krb5kdc[24602](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790492547, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 07:02:27.836982 25158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:02:27.837453 25158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:02:27.837659 25158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:02:27.849469 25158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:02:27.849628 25158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:02:27.849896 25158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:02:27.866022 25158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.121.212:38071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:45123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:02:27.867869 25158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:02:27.869598 25158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:02:27.887483 25168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:02:27.888077 25169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:02:27.897511 25158 server_base.cc:1034] running on GCE node
W20260927 07:02:27.897985 25171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:02:27.898862 25158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:02:27.904707 25158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:02:27.906101 25158 hybrid_clock.cc:648] HybridClock initialized: now 1790492547905981 us; error 110 us; skew 500 ppm
I20260927 07:02:27.906562 25158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:02:27 dist-test-slave-p6hz krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492547, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:02:27.910974 25158 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:02:27.913170 25158 webserver.cc:460] Webserver started at http://127.13.121.193:46315/ using document root <none> and password file <none>
I20260927 07:02:27.914559 25158 fs_manager.cc:359] Metadata directory not provided
I20260927 07:02:27.914713 25158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:02:27.915105 25158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:02:27 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492547, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:02:27.935411 25158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "87412918c726488db03c9d8e4839cd11"
format_stamp: "Formatted at 2026-09-27 07:02:27 on dist-test-slave-p6hz"
server_key: "c42eeeb3dde60a83819b006daff81115"
server_key_iv: "30e1d31d55a23340f67e9dd64cf01689"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:27.936328 25158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "87412918c726488db03c9d8e4839cd11"
format_stamp: "Formatted at 2026-09-27 07:02:27 on dist-test-slave-p6hz"
server_key: "c42eeeb3dde60a83819b006daff81115"
server_key_iv: "30e1d31d55a23340f67e9dd64cf01689"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:27.941557 25158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:02:27.963804 25178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:02:27.965466 25158 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.008s sys 0.000s
I20260927 07:02:27.965664 25158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "87412918c726488db03c9d8e4839cd11"
format_stamp: "Formatted at 2026-09-27 07:02:27 on dist-test-slave-p6hz"
server_key: "c42eeeb3dde60a83819b006daff81115"
server_key_iv: "30e1d31d55a23340f67e9dd64cf01689"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:27.965901 25158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:02:28.042424 25158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:02:28.048053 25158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:02:28.048494 25158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:02:28.056020 25158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:02:28.059935 25158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:02:28.060058 25158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:02:28.060192 25158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:02:28.060271 25158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:02:28 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492547, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:02:28.192790 25158 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:45429
I20260927 07:02:28.192854 25290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:45429 every 8 connection(s)
I20260927 07:02:28.194581 25158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:02:28.202669 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 25158
I20260927 07:02:28.202898 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:02:28.203769 13799 external_mini_cluster.cc:1428] Setting key ee04c499f7cc20a9abb12a4785d23b3f
Sep 27 07:02:28 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492547, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:02:28.247526 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:28.205385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:58931 (local address 127.13.121.254:45123)
0927 07:02:28.206434 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:28.206440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:28.218862 (+ 12422us) server_negotiation.cc:407] Connection header received
0927 07:02:28.225172 (+ 6310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:28.225179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:28.225286 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:28.225360 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:02:28.227627 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:28.228692 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:28.230999 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:28.231616 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:28.231709 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:28.236135 (+ 4426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:02:28.236143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:02:28.236148 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:02:28.236192 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:02:28.239494 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:28.244488 (+ 4994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:28.244495 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:28.244498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:28.244677 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:28.245325 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:28.245329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:28.245332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:28.245691 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:02:28.245813 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:28.246881 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0927 07:02:28.247123 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":774,"threads_started":1}
I20260927 07:02:28.250056 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:28.206174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:45123 (local address 127.13.121.193:58931)
0927 07:02:28.218652 (+ 12478us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:28.218727 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0927 07:02:28.224798 (+ 6071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:02:28.225929 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:02:28.225942 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:02:28.226540 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:02:28.227423 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:28.227452 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:28.228902 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:28.228907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:28.230829 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:28.230845 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:28.232069 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:28.232076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:28.232361 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:28.233121 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:02:28.233150 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:02:28.235832 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:02:28.239707 (+ 3875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:02:28.239716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:02:28.239736 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:02:28.240363 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:02:28.244846 (+ 4483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:02:28.244851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:02:28.244854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:02:28.245171 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:02:28.245914 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:02:28.245923 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:02:28.246431 (+ 508us) client_negotiation.cc:769] Sending connection context
0927 07:02:28.248075 (+ 1644us) client_negotiation.cc:240] Negotiation successful
0927 07:02:28.248654 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12119,"thread_start_us":12586,"threads_started":1}
I20260927 07:02:28.252736 25291 heartbeater.cc:347] Connected to a master server at 127.13.121.254:45123
I20260927 07:02:28.253099 25291 heartbeater.cc:464] Registering TS with master...
I20260927 07:02:28.254025 25291 heartbeater.cc:511] Master 127.13.121.254:45123 requested a full tablet report, sending...
I20260927 07:02:28.256773 25078 ts_manager.cc:194] Registered new tserver with Master: 87412918c726488db03c9d8e4839cd11 (127.13.121.193:45429)
I20260927 07:02:28.258848 25078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:58931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:02:28.303118 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:45123
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:02:28 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492547, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
W20260927 07:02:28.800196 25297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:02:28.800915 25297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:02:28.801133 25297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:02:28.812885 25297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:02:28.813110 25297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:02:28.813330 25297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:02:28.828897 25297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.121.212:38071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:45123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:02:28.830889 25297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:02:28.832780 25297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:02:28.855397 25306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:02:28.857112 25307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:02:28.860932 25314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:02:28.861274 25297 server_base.cc:1034] running on GCE node
I20260927 07:02:28.861954 25297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:02:28.862965 25297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:02:28.864181 25297 hybrid_clock.cc:648] HybridClock initialized: now 1790492548864116 us; error 63 us; skew 500 ppm
I20260927 07:02:28.864501 25297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:02:28 dist-test-slave-p6hz krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492548, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:02:28.867681 25297 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:02:28.869302 25297 webserver.cc:460] Webserver started at http://127.13.121.194:46753/ using document root <none> and password file <none>
I20260927 07:02:28.870044 25297 fs_manager.cc:359] Metadata directory not provided
I20260927 07:02:28.870164 25297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:02:28.870481 25297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:02:28 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492548, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:02:28.889382 25297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "4064737170f445c7a78853bcc2ec3086"
format_stamp: "Formatted at 2026-09-27 07:02:28 on dist-test-slave-p6hz"
server_key: "f8e26642eda3e97f5ae7a69c0e74695d"
server_key_iv: "d5358e91f0c1de26de22f304b83cc03a"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:28.890419 25297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "4064737170f445c7a78853bcc2ec3086"
format_stamp: "Formatted at 2026-09-27 07:02:28 on dist-test-slave-p6hz"
server_key: "f8e26642eda3e97f5ae7a69c0e74695d"
server_key_iv: "d5358e91f0c1de26de22f304b83cc03a"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:28.897478 25297 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260927 07:02:28.919828 25321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:02:28.921895 25297 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.004s
I20260927 07:02:28.922178 25297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "4064737170f445c7a78853bcc2ec3086"
format_stamp: "Formatted at 2026-09-27 07:02:28 on dist-test-slave-p6hz"
server_key: "f8e26642eda3e97f5ae7a69c0e74695d"
server_key_iv: "d5358e91f0c1de26de22f304b83cc03a"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:28.922524 25297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:02:28.948513 25297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:02:28.953946 25297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:02:28.954360 25297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:02:28.955801 25297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:02:28.957649 25297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:02:28.957801 25297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260927 07:02:28.957955 25297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:02:28.958038 25297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:02:29.006580 25297 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:35393
I20260927 07:02:29.006505 25433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:35393 every 8 connection(s)
I20260927 07:02:29.008255 25297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:02:29.017186 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 25297
I20260927 07:02:29.017387 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:02:29.017976 13799 external_mini_cluster.cc:1428] Setting key d2c84c68c789c35570cd8cb6245e4377
Sep 27 07:02:29 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492548, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:02:29.046286 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.012371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:55075 (local address 127.13.121.254:45123)
0927 07:02:29.023209 (+ 10838us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.023215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.023235 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:02:29.023312 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.023318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.023399 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.023499 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:02:29.025670 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.026824 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.029296 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.029950 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.030050 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.038815 (+ 8765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:02:29.038823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:02:29.038828 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:02:29.038890 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:02:29.042466 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.043604 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.043610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.043614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.043773 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.044318 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.044324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.044327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.044721 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:02:29.044882 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.045643 (+ 761us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.045904 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10678,"thread_start_us":10559,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
I20260927 07:02:29.047036 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.011739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:45123 (local address 127.13.121.194:55075)
0927 07:02:29.019761 (+ 8022us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:29.019835 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0927 07:02:29.022348 (+ 2513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:02:29.023968 (+ 1620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:02:29.023983 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:02:29.024599 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:02:29.025444 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.025474 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.027040 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.027045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.029118 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.029133 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.030393 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.030400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.030703 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.035490 (+ 4787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:02:29.035530 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:02:29.038463 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:02:29.042673 (+ 4210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:02:29.042686 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:02:29.042708 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:02:29.043439 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:02:29.043899 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:02:29.043904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:02:29.043908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:02:29.044178 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:02:29.044841 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:02:29.044848 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:02:29.045214 (+ 366us) client_negotiation.cc:769] Sending connection context
0927 07:02:29.045475 (+ 261us) client_negotiation.cc:240] Negotiation successful
0927 07:02:29.045878 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7695,"thread_start_us":7461,"threads_started":1}
I20260927 07:02:29.049054 25434 heartbeater.cc:347] Connected to a master server at 127.13.121.254:45123
I20260927 07:02:29.049355 25434 heartbeater.cc:464] Registering TS with master...
I20260927 07:02:29.050040 25434 heartbeater.cc:511] Master 127.13.121.254:45123 requested a full tablet report, sending...
I20260927 07:02:29.051476 25078 ts_manager.cc:194] Registered new tserver with Master: 4064737170f445c7a78853bcc2ec3086 (127.13.121.194:35393)
I20260927 07:02:29.052268 25078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:55075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:02:29.075513 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:45123
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:02:29.262941 25291 heartbeater.cc:503] Master 127.13.121.254:45123 was elected leader, sending a full tablet report...
W20260927 07:02:29.410816 25440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:02:29.411087 25440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:02:29.411226 25440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:02:29.418526 25440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:02:29.418649 25440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:02:29.418790 25440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:02:29.428996 25440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.121.212:38071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:45123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:02:29.430363 25440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:02:29.431706 25440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:02:29.447187 25446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:02:29.447474 25445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:02:29.447786 25448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:02:29.448170 25440 server_base.cc:1034] running on GCE node
I20260927 07:02:29.448777 25440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:02:29.449918 25440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:02:29.451249 25440 hybrid_clock.cc:648] HybridClock initialized: now 1790492549451095 us; error 54 us; skew 500 ppm
I20260927 07:02:29.451593 25440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:02:29 dist-test-slave-p6hz krb5kdc[24602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492549, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:02:29.454838 25440 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:02:29.456584 25440 webserver.cc:460] Webserver started at http://127.13.121.195:35747/ using document root <none> and password file <none>
I20260927 07:02:29.457289 25440 fs_manager.cc:359] Metadata directory not provided
I20260927 07:02:29.457405 25440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:02:29.457705 25440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:02:29 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492549, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:02:29.474115 25440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "4f079285af904afb89e3285dc9bf811e"
format_stamp: "Formatted at 2026-09-27 07:02:29 on dist-test-slave-p6hz"
server_key: "5172b58bd3b4876ab0c247e757e15cdf"
server_key_iv: "b9867a0c77f4811424abee258ade5531"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:29.474977 25440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "4f079285af904afb89e3285dc9bf811e"
format_stamp: "Formatted at 2026-09-27 07:02:29 on dist-test-slave-p6hz"
server_key: "5172b58bd3b4876ab0c247e757e15cdf"
server_key_iv: "b9867a0c77f4811424abee258ade5531"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:29.480624 25440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260927 07:02:29.496833 25455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:02:29.498340 25440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:02:29.498528 25440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "4f079285af904afb89e3285dc9bf811e"
format_stamp: "Formatted at 2026-09-27 07:02:29 on dist-test-slave-p6hz"
server_key: "5172b58bd3b4876ab0c247e757e15cdf"
server_key_iv: "b9867a0c77f4811424abee258ade5531"
server_key_version: "kuduclusterkey@0"
I20260927 07:02:29.498760 25440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:02:29.532006 25440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:02:29.536533 25440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:02:29.536859 25440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:02:29.537899 25440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:02:29.539343 25440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:02:29.539424 25440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:02:29.539541 25440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:02:29.539594 25440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:02:29.574301 25440 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:38027
I20260927 07:02:29.574337 25567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:38027 every 8 connection(s)
I20260927 07:02:29.575681 25440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:02:29.577396 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 25440
I20260927 07:02:29.577553 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:02:29.578008 13799 external_mini_cluster.cc:1428] Setting key 7b589fa1f99ead409ae86dcd7dcb76f5
Sep 27 07:02:29 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492549, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:02:29.601078 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.578840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:54855 (local address 127.13.121.254:45123)
0927 07:02:29.582121 (+ 3281us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.582128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.583921 (+ 1793us) server_negotiation.cc:407] Connection header received
0927 07:02:29.585712 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.585718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.585805 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.585926 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:02:29.587364 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.588504 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.590123 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.590775 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.590911 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.594678 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:02:29.594685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:02:29.594690 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:02:29.594731 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:02:29.598053 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.598835 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.598841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.598845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.599029 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.599488 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.599493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.599497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.599835 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:02:29.599984 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.600631 (+ 647us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.600817 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3141,"thread_start_us":2971,"threads_started":1}
I20260927 07:02:29.602083 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.579761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:45123 (local address 127.13.121.195:54855)
0927 07:02:29.583748 (+ 3987us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:29.583799 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 07:02:29.585454 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:02:29.586099 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:02:29.586108 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:02:29.586540 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:02:29.587187 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.587214 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.588694 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.588698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.589974 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.589988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.591021 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.591026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.591319 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.592002 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:02:29.592030 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:02:29.594478 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:02:29.598212 (+ 3734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:02:29.598219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:02:29.598235 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:02:29.598705 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:02:29.599120 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:02:29.599123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:02:29.599125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:02:29.599386 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:02:29.599947 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:02:29.599953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:02:29.600318 (+ 365us) client_negotiation.cc:769] Sending connection context
0927 07:02:29.600546 (+ 228us) client_negotiation.cc:240] Negotiation successful
0927 07:02:29.600934 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3676,"thread_start_us":3422,"threads_started":1}
I20260927 07:02:29.603667 25568 heartbeater.cc:347] Connected to a master server at 127.13.121.254:45123
I20260927 07:02:29.603996 25568 heartbeater.cc:464] Registering TS with master...
I20260927 07:02:29.604679 25568 heartbeater.cc:511] Master 127.13.121.254:45123 requested a full tablet report, sending...
I20260927 07:02:29.606065 25078 ts_manager.cc:194] Registered new tserver with Master: 4f079285af904afb89e3285dc9bf811e (127.13.121.195:38027)
I20260927 07:02:29.606945 25078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:54855
I20260927 07:02:29.618732 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:02:29.635993 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.623268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.13.121.254:45123)
0927 07:02:29.623425 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.623429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.623840 (+ 411us) server_negotiation.cc:407] Connection header received
0927 07:02:29.623961 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.623965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.624049 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.624135 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:02:29.625564 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.626307 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.627733 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.628234 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.628327 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.630740 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:02:29.630745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:02:29.630748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:02:29.630787 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:02:29.633388 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.634035 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.634038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.634041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.634154 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.634532 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.634535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.634537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.634818 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:02:29.634996 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.635496 (+ 500us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.635690 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:02:29.640051 25078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:02:29.646967 25078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:02:29.683786 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.659338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.13.121.194:35393)
0927 07:02:29.674884 (+ 15546us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.674893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.674921 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:02:29.675016 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.675024 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.675361 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.675557 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:02:29.676507 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.678122 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.680908 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.682527 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.682688 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.682884 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.683274 (+ 390us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.683448 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15399,"thread_start_us":15282,"threads_started":1}
I20260927 07:02:29.684000 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.659113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:35393 (local address 127.0.0.1:36104)
0927 07:02:29.672330 (+ 13217us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:29.672398 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0927 07:02:29.672611 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:02:29.675843 (+ 3232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:02:29.675851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:02:29.675880 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:02:29.676301 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.676314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.678298 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.678304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.680745 (+ 2441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.680760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.682780 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.682786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.682981 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.683034 (+ 53us) client_negotiation.cc:769] Sending connection context
0927 07:02:29.683231 (+ 197us) client_negotiation.cc:240] Negotiation successful
0927 07:02:29.683561 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12876,"mutex_wait_us":2,"thread_start_us":15144,"threads_started":1}
I20260927 07:02:29.686607 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.659739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39786 (local address 127.13.121.195:38027)
0927 07:02:29.678704 (+ 18965us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.678712 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.678751 (+ 39us) server_negotiation.cc:407] Connection header received
0927 07:02:29.678832 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.678840 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.679056 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.679376 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:02:29.680021 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.681323 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.684135 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.685552 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.685745 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.685913 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.686120 (+ 207us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.686282 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18795,"thread_start_us":18636,"threads_started":1}
I20260927 07:02:29.686623 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.659425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:38027 (local address 127.0.0.1:39786)
0927 07:02:29.674100 (+ 14675us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:29.674119 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:02:29.674241 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:02:29.679277 (+ 5036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:02:29.679282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:02:29.679312 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:02:29.679836 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.679849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.681485 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.681489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.684007 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.684020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.685758 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.685764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.685928 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.685962 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 07:02:29.686073 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 07:02:29.686210 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14541,"mutex_wait_us":76,"thread_start_us":13416,"threads_started":1}
I20260927 07:02:29.687273 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.674209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.13.121.193:45429)
0927 07:02:29.678066 (+ 3857us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.678075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.678498 (+ 423us) server_negotiation.cc:407] Connection header received
0927 07:02:29.678608 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.678617 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.678852 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.679038 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:02:29.680048 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.681488 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.684232 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.685573 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.685745 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.685913 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.686742 (+ 829us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.686906 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3685,"thread_start_us":3274,"threads_started":1}
I20260927 07:02:29.687343 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.674978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45429 (local address 127.0.0.1:45746)
0927 07:02:29.678213 (+ 3235us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:29.678231 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:02:29.678371 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:02:29.679029 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:02:29.679034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:02:29.679058 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:02:29.679837 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.679849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.681649 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.681654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.684051 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.684067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.686065 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.686071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.686255 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.686287 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:02:29.686367 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 07:02:29.686498 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3106,"mutex_wait_us":70,"thread_start_us":3111,"threads_started":1}
I20260927 07:02:29.688145 25369 tablet_service.cc:1467] Processing CreateTablet for tablet ffa57447417443248b8584cd82d453a9 (DEFAULT_TABLE table=test-table [id=67418f9f2d7c445bb9582ce2e97fb0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:02:29.689638 25369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa57447417443248b8584cd82d453a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:02:29.690310 25503 tablet_service.cc:1467] Processing CreateTablet for tablet ffa57447417443248b8584cd82d453a9 (DEFAULT_TABLE table=test-table [id=67418f9f2d7c445bb9582ce2e97fb0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:02:29.690847 25226 tablet_service.cc:1467] Processing CreateTablet for tablet ffa57447417443248b8584cd82d453a9 (DEFAULT_TABLE table=test-table [id=67418f9f2d7c445bb9582ce2e97fb0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:02:29.691699 25503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa57447417443248b8584cd82d453a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:02:29.692196 25226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa57447417443248b8584cd82d453a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:02:29.700410 25587 tablet_bootstrap.cc:492] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: Bootstrap starting.
I20260927 07:02:29.703832 25587 tablet_bootstrap.cc:654] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: Neither blocks nor log segments found. Creating new log.
I20260927 07:02:29.704835 25587 log.cc:824] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: Log is configured to *not* fsync() on all Append() calls
I20260927 07:02:29.706007 25589 tablet_bootstrap.cc:492] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: Bootstrap starting.
I20260927 07:02:29.706840 25588 tablet_bootstrap.cc:492] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11: Bootstrap starting.
I20260927 07:02:29.709460 25587 tablet_bootstrap.cc:492] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: No bootstrap required, opened a new log
I20260927 07:02:29.709800 25587 ts_tablet_manager.cc:1397] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260927 07:02:29.710608 25588 tablet_bootstrap.cc:654] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11: Neither blocks nor log segments found. Creating new log.
I20260927 07:02:29.711027 25589 tablet_bootstrap.cc:654] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: Neither blocks nor log segments found. Creating new log.
I20260927 07:02:29.711910 25588 log.cc:824] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11: Log is configured to *not* fsync() on all Append() calls
I20260927 07:02:29.712147 25589 log.cc:824] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:02:29.715008 25588 tablet_bootstrap.cc:492] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11: No bootstrap required, opened a new log
I20260927 07:02:29.715008 25589 tablet_bootstrap.cc:492] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: No bootstrap required, opened a new log
I20260927 07:02:29.715315 25588 ts_tablet_manager.cc:1397] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260927 07:02:29.715322 25589 ts_tablet_manager.cc:1397] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260927 07:02:29.719400 25587 raft_consensus.cc:348] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } }
I20260927 07:02:29.719923 25587 raft_consensus.cc:374] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:02:29.720049 25587 raft_consensus.cc:729] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4064737170f445c7a78853bcc2ec3086, State: Initialized, Role: FOLLOWER
I20260927 07:02:29.720876 25587 consensus_queue.cc:260] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } }
I20260927 07:02:29.723920 25434 heartbeater.cc:503] Master 127.13.121.254:45123 was elected leader, sending a full tablet report...
I20260927 07:02:29.724619 25587 ts_tablet_manager.cc:1428] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260927 07:02:29.726217 25589 raft_consensus.cc:348] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } }
I20260927 07:02:29.726567 25589 raft_consensus.cc:374] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:02:29.726658 25589 raft_consensus.cc:729] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f079285af904afb89e3285dc9bf811e, State: Initialized, Role: FOLLOWER
I20260927 07:02:29.727306 25589 consensus_queue.cc:260] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } }
I20260927 07:02:29.729046 25588 raft_consensus.cc:348] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } }
I20260927 07:02:29.729552 25588 raft_consensus.cc:374] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:02:29.729738 25588 raft_consensus.cc:729] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87412918c726488db03c9d8e4839cd11, State: Initialized, Role: FOLLOWER
I20260927 07:02:29.730440 25588 consensus_queue.cc:260] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } }
I20260927 07:02:29.731182 25568 heartbeater.cc:503] Master 127.13.121.254:45123 was elected leader, sending a full tablet report...
I20260927 07:02:29.731982 25589 ts_tablet_manager.cc:1428] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260927 07:02:29.732131 25588 ts_tablet_manager.cc:1428] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11: Time spent starting tablet: real 0.017s user 0.013s sys 0.000s
I20260927 07:02:29.738756 25595 raft_consensus.cc:482] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:02:29.739005 25595 raft_consensus.cc:504] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } }
I20260927 07:02:29.740628 25595 leader_election.cc:288] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4064737170f445c7a78853bcc2ec3086 (127.13.121.194:35393), 4f079285af904afb89e3285dc9bf811e (127.13.121.195:38027)
I20260927 07:02:29.747315 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.741013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:53291 (local address 127.13.121.195:38027)
0927 07:02:29.741206 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.741212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.741979 (+ 767us) server_negotiation.cc:407] Connection header received
0927 07:02:29.742152 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.742156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.742211 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.742284 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:02:29.743028 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.743800 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.745597 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.746358 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.746506 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.746584 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.746863 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.747046 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 07:02:29.747334 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.741129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:36351 (local address 127.13.121.194:35393)
0927 07:02:29.741326 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.741333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.742002 (+ 669us) server_negotiation.cc:407] Connection header received
0927 07:02:29.742205 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.742210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.742280 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.742351 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:02:29.743171 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.743934 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.745686 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.746465 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.746601 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.746672 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.746952 (+ 280us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.747074 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 07:02:29.747469 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.740971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:38027 (local address 127.13.121.193:53291)
0927 07:02:29.741856 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:29.741875 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:02:29.742028 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:02:29.742379 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:02:29.742383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:02:29.742440 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:02:29.742809 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.742820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.743935 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.743940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.745448 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.745461 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.746548 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.746555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.746720 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.746746 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 07:02:29.746827 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 07:02:29.746959 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":32,"spinlock_wait_cycles":33536,"thread_start_us":475,"threads_started":1}
I20260927 07:02:29.747749 25597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.740995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:35393 (local address 127.13.121.193:36351)
0927 07:02:29.741902 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:29.741917 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:02:29.742079 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:02:29.742376 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:02:29.742380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:02:29.742404 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:02:29.742970 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.742983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.744071 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.744080 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.745541 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:02:29.745553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.746677 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.746684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:02:29.746815 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.746837 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 07:02:29.746914 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 07:02:29.747035 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":4,"spinlock_wait_cycles":22912,"thread_start_us":548,"threads_started":1}
I20260927 07:02:29.748445 25523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa57447417443248b8584cd82d453a9" candidate_uuid: "87412918c726488db03c9d8e4839cd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f079285af904afb89e3285dc9bf811e" is_pre_election: true
I20260927 07:02:29.748885 25523 raft_consensus.cc:2454] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87412918c726488db03c9d8e4839cd11 in term 0.
I20260927 07:02:29.749439 25389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa57447417443248b8584cd82d453a9" candidate_uuid: "87412918c726488db03c9d8e4839cd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4064737170f445c7a78853bcc2ec3086" is_pre_election: true
I20260927 07:02:29.749552 25180 leader_election.cc:302] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f079285af904afb89e3285dc9bf811e, 87412918c726488db03c9d8e4839cd11; no voters:
I20260927 07:02:29.749969 25595 raft_consensus.cc:2781] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:02:29.749941 25389 raft_consensus.cc:2454] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87412918c726488db03c9d8e4839cd11 in term 0.
I20260927 07:02:29.750101 25595 raft_consensus.cc:482] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:02:29.750192 25595 raft_consensus.cc:3037] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:02:29.752593 25595 raft_consensus.cc:504] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } }
I20260927 07:02:29.753206 25595 leader_election.cc:288] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [CANDIDATE]: Term 1 election: Requested vote from peers 4064737170f445c7a78853bcc2ec3086 (127.13.121.194:35393), 4f079285af904afb89e3285dc9bf811e (127.13.121.195:38027)
I20260927 07:02:29.753690 25389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa57447417443248b8584cd82d453a9" candidate_uuid: "87412918c726488db03c9d8e4839cd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4064737170f445c7a78853bcc2ec3086"
I20260927 07:02:29.753728 25523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa57447417443248b8584cd82d453a9" candidate_uuid: "87412918c726488db03c9d8e4839cd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f079285af904afb89e3285dc9bf811e"
I20260927 07:02:29.753907 25389 raft_consensus.cc:3037] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:02:29.753921 25523 raft_consensus.cc:3037] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:02:29.757056 25523 raft_consensus.cc:2454] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87412918c726488db03c9d8e4839cd11 in term 1.
I20260927 07:02:29.757055 25389 raft_consensus.cc:2454] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87412918c726488db03c9d8e4839cd11 in term 1.
I20260927 07:02:29.757491 25179 leader_election.cc:302] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4064737170f445c7a78853bcc2ec3086, 87412918c726488db03c9d8e4839cd11; no voters:
I20260927 07:02:29.757831 25595 raft_consensus.cc:2781] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:02:29.758383 25595 raft_consensus.cc:686] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 1 LEADER]: Becoming Leader. State: Replica: 87412918c726488db03c9d8e4839cd11, State: Running, Role: LEADER
I20260927 07:02:29.758852 25595 consensus_queue.cc:237] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } }
I20260927 07:02:29.763442 25078 catalog_manager.cc:5495] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87412918c726488db03c9d8e4839cd11 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "87412918c726488db03c9d8e4839cd11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87412918c726488db03c9d8e4839cd11" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45429 } health_report { overall_health: HEALTHY } } }
W20260927 07:02:29.766316 25435 tablet.cc:2369] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:02:29.794777 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.789843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.13.121.193:45429)
0927 07:02:29.790005 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.790010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.790027 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:02:29.790203 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.790207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.790297 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.790415 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:02:29.790772 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.791585 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.792919 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.793304 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.793390 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.793815 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:02:29.794347 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:02:29.794453 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.794512 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.794578 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:02:29.802878 25523 raft_consensus.cc:1261] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [term 1 FOLLOWER]: Refusing update from remote peer 87412918c726488db03c9d8e4839cd11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:02:29.802878 25389 raft_consensus.cc:1261] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [term 1 FOLLOWER]: Refusing update from remote peer 87412918c726488db03c9d8e4839cd11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:02:29.804000 25598 consensus_queue.cc:1035] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4064737170f445c7a78853bcc2ec3086" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:02:29.804340 25595 consensus_queue.cc:1035] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f079285af904afb89e3285dc9bf811e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:02:29.815253 25603 mvcc.cc:205] Tried to move back new op lower bound from 7333857483982315520 to 7333857483815862272. Current Snapshot: MvccSnapshot[applied={T|T < 7333857483982315520}]
I20260927 07:02:29.827515 25078 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:02:29.827905 25078 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:02:29.831650 25078 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89f2c67de62049548aa3c849dbe8f6b6: Sending DeleteTablet for 3 replicas of tablet ffa57447417443248b8584cd82d453a9
I20260927 07:02:29.832715 25369 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa57447417443248b8584cd82d453a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:02:29 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:36104
I20260927 07:02:29.832834 25226 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa57447417443248b8584cd82d453a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:02:29 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:45746
I20260927 07:02:29.833492 25503 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa57447417443248b8584cd82d453a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:02:29 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:39786
I20260927 07:02:29.834525 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 25158
Sep 27 07:02:29 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:02:29.858618 25612 tablet_replica.cc:330] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: stopping tablet replica
I20260927 07:02:29.859332 25612 raft_consensus.cc:2229] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:02:29.860034 25611 tablet_replica.cc:330] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11: stopping tablet replica
I20260927 07:02:29.860610 25612 raft_consensus.cc:2258] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:02:29.861141 25611 raft_consensus.cc:2229] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:02:29.861907 25611 raft_consensus.cc:2258] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:02:29.864367 25612 ts_tablet_manager.cc:1905] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:02:29.864398 25611 ts_tablet_manager.cc:1905] T ffa57447417443248b8584cd82d453a9 P 87412918c726488db03c9d8e4839cd11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:02:29.867010 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.835879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45774 (local address 127.13.121.193:45429)
0927 07:02:29.836056 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.836063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.846347 (+ 10284us) server_negotiation.cc:407] Connection header received
0927 07:02:29.846867 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.846873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.846964 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.847041 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:02:29.848947 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.850266 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.852755 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.853480 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.853588 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.859182 (+ 5594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:02:29.859206 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:02:29.859226 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:02:29.859292 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:02:29.863237 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.864143 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.864152 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.864156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.864391 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.864847 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.864851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.864854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.865337 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:02:29.865765 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.866373 (+ 608us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.866628 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:02:29.868093 25610 tablet_replica.cc:330] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: stopping tablet replica
I20260927 07:02:29.868536 25286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:02:29.870036 25610 raft_consensus.cc:2229] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:02:29.871462 25612 ts_tablet_manager.cc:1918] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:02:29.871619 25612 log.cc:1196] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/ffa57447417443248b8584cd82d453a9
I20260927 07:02:29.872110 25612 ts_tablet_manager.cc:1939] T ffa57447417443248b8584cd82d453a9 P 4f079285af904afb89e3285dc9bf811e: Deleting consensus metadata
I20260927 07:02:29.871057 25610 raft_consensus.cc:2258] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:02:29.874294 25610 ts_tablet_manager.cc:1905] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:02:29.874519 25063 catalog_manager.cc:4841] TS 4f079285af904afb89e3285dc9bf811e (127.13.121.195:38027): tablet ffa57447417443248b8584cd82d453a9 (table test-table [id=67418f9f2d7c445bb9582ce2e97fb0fa]) successfully deleted
I20260927 07:02:29.878973 25610 ts_tablet_manager.cc:1918] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:02:29.879107 25610 log.cc:1196] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/ffa57447417443248b8584cd82d453a9
I20260927 07:02:29.879563 25610 ts_tablet_manager.cc:1939] T ffa57447417443248b8584cd82d453a9 P 4064737170f445c7a78853bcc2ec3086: Deleting consensus metadata
I20260927 07:02:29.881171 25062 catalog_manager.cc:4841] TS 4064737170f445c7a78853bcc2ec3086 (127.13.121.194:35393): tablet ffa57447417443248b8584cd82d453a9 (table test-table [id=67418f9f2d7c445bb9582ce2e97fb0fa]) successfully deleted
I20260927 07:02:29.966713 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 25158
I20260927 07:02:29.981156 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 25297
W20260927 07:02:29.981331 25063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.121.193:45429 (error 108)
I20260927 07:02:29.983254 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.982729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45429 (local address 127.0.0.1:45778)
0927 07:02:29.982927 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:29.983040 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260927 07:02:29.983484 25063 catalog_manager.cc:4584] TS 87412918c726488db03c9d8e4839cd11 (127.13.121.193:45429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ffa57447417443248b8584cd82d453a9: Network error: Client connection negotiation failed: client connection to 127.13.121.193:45429: connect: Connection refused (error 111)
Sep 27 07:02:29 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:02:29.996052 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:29.981962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36108 (local address 127.13.121.194:35393)
0927 07:02:29.982136 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:29.982142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:29.982277 (+ 135us) server_negotiation.cc:407] Connection header received
0927 07:02:29.982481 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:29.982485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:29.982564 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:29.982662 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:02:29.984126 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.984812 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.986125 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:29.986497 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:29.986581 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:29.990618 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:02:29.990632 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:02:29.990648 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:02:29.990691 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:02:29.993328 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.994218 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.994226 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.994228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.994373 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:29.994659 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:29.994662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:29.994664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:29.995016 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:02:29.995322 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:29.995611 (+ 289us) server_negotiation.cc:299] Negotiation successful
0927 07:02:29.995776 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 07:02:29.996642 25429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:02:30.037146 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:30.036575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45429 (local address 127.0.0.1:45792)
0927 07:02:30.036878 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:30.036984 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260927 07:02:30.038173 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:30.037861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45429 (local address 127.0.0.1:45802)
0927 07:02:30.037983 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:30.038066 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 07:02:30.083803 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 25297
I20260927 07:02:30.097188 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 25440
I20260927 07:02:30.098778 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:30.098221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45429 (local address 127.0.0.1:45806)
0927 07:02:30.098493 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:30.098583 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260927 07:02:30.099912 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:30.099517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45429 (local address 127.0.0.1:45820)
0927 07:02:30.099702 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:30.099786 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
Sep 27 07:02:30 dist-test-slave-p6hz krb5kdc[24602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:02:30.114212 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:30.098206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.13.121.195:38027)
0927 07:02:30.098457 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 07:02:30.098464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:02:30.098542 (+ 78us) server_negotiation.cc:407] Connection header received
0927 07:02:30.098827 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:02:30.098833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:02:30.098982 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:02:30.099109 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:02:30.100872 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:30.101989 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:30.103934 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:02:30.104326 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:02:30.104440 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:02:30.108535 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:02:30.108550 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:02:30.108566 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:02:30.108604 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:02:30.111301 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:30.112091 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:30.112099 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:30.112101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:30.112243 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:02:30.112676 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:02:30.112680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:02:30.112682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:02:30.113063 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:02:30.113295 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0927 07:02:30.113714 (+ 419us) server_negotiation.cc:299] Negotiation successful
0927 07:02:30.113893 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:02:30.114785 25563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:02:30.180372 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:30.179848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45429 (local address 127.0.0.1:45822)
0927 07:02:30.180053 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:30.180158 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260927 07:02:30.181535 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:02:30.181091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45429 (local address 127.0.0.1:45836)
0927 07:02:30.181345 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0927 07:02:30.181421 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:45429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260927 07:02:30.202921 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 25440
I20260927 07:02:30.215783 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 25046
I20260927 07:02:30.216876 25108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:02:30.329381 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 25046
I20260927 07:02:30.347464 13799 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 07:02:30.683455 13799 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 07:02:30.683668 13799 mini_ranger.cc:66] Stopping Ranger...
I20260927 07:02:31.019377 13799 mini_ranger.cc:68] Stopped Ranger
2026-09-27 07:02:31.019 UTC [24632] LOG: received smart shutdown request
2026-09-27 07:02:31.023 UTC [24632] LOG: background worker "logical replication launcher" (PID 24641) exited with exit code 1
2026-09-27 07:02:31.023 UTC [24636] LOG: shutting down
2026-09-27 07:02:31.215 UTC [24632] LOG: database system is shut down
2026-09-27T07:02:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50264 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:02:31 dist-test-slave-p6hz krb5kdc[25627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:02:31 dist-test-slave-p6hz krb5kdc[25627](info): set up 2 sockets
Sep 27 07:02:31 dist-test-slave-p6hz krb5kdc[25627](info): commencing operation
krb5kdc: starting...
W20260927 07:02:32.347559 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:02:32 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:02:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:02:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.121.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.121.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/rangeradmin_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/rangeradmin_127.13.121.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.121.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.121.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/rangerlookup_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/rangerlookup_127.13.121.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.121.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.121.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/HTTP_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/HTTP_127.13.121.212@KRBTEST.COM.keytab.
I20260927 07:02:32.660847 13799 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task8NDO8F/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/postgres -l logfile start
2026-09-27 07:02:35.258 UTC [25657] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 07:02:35.258 UTC [25657] LOG: listening on IPv4 address "127.13.121.212", port 43745
2026-09-27 07:02:35.262 UTC [25657] LOG: listening on Unix socket "/tmp/.s.PGSQL.43745"
2026-09-27 07:02:35.283 UTC [25660] LOG: database system was shut down at 2026-09-27 07:02:33 UTC
2026-09-27 07:02:35.286 UTC [25657] LOG: database system is ready to accept connections
I20260927 07:02:36.171236 13799 mini_postgres.cc:96] Postgres bound to 43745
2026-09-27 07:02:36.187 UTC [25668] FATAL: database "slave" does not exist
127.13.121.212:43745 - accepting connections
I20260927 07:02:36.187674 13799 mini_ranger.cc:161] Starting Ranger...
I20260927 07:02:36.212687 13799 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 07:02:36.719323 13799 mini_ranger.cc:87] Created ranger Postgres database
I20260927 07:02:36.719506 13799 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-admin
2026-09-27 07:02:37,023 [I] DB FLAVOR :POSTGRES
2026-09-27 07:02:37,024 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 07:02:37,024 [I] Checking connection..
2026-09-27 07:02:37,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 07:02:37,277 [I] Checking connection passed.
2026-09-27 07:02:37,277 [I] --------- Verifying version history table ---------
2026-09-27 07:02:37,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:02:37,538 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 07:02:37,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:02:37,797 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 07:02:37,797 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 07:02:37,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 07:02:38.054 UTC [25732] WARNING: there is no transaction in progress
2026-09-27 07:02:38,061 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 07:02:38,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 07:02:38,294 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 07:02:38,294 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 07:02:38,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 07:02:38,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 07:02:38,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p6hz.c.gcp-upstream.internal','N') ;"
2026-09-27 07:02:39,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 07:02:39,307 [I] Table x_portal_user does not exist in database ranger
2026-09-27 07:02:39,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 07:02:39,541 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 07:02:39,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 07:02:39,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 07:02:40,052 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 07:02:40,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 07:02:41.075 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.092 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.110 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.128 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.146 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.168 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.230 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.238 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.255 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.266 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.278 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.293 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.303 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.311 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.318 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:41.326 UTC [25840] WARNING: there is no transaction in progress
2026-09-27 07:02:42,057 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 07:02:42,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 07:02:42,298 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 07:02:42,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 07:02:42,551 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 07:02:42,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 07:02:42,796 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 07:02:42,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 07:02:43,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 07:02:43,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p6hz.c.gcp-upstream.internal';"
2026-09-27 07:02:43,517 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 07:02:43,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 07:02:43,754 [I] DB_PATCHES have already been applied
I20260927 07:02:43.760702 13799 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-admin:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task8NDO8F/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 07:02:43.760807 13799 mini_ranger.cc:193] Using host: 127.13.121.212
I20260927 07:02:43.774031 13799 mini_ranger.cc:227] Ranger admin URL: http://127.13.121.212:51167
Sep 27, 2026 7:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 7:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 7:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 7:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 7:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 7:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 7:02:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 07:02:44.800933 13799 mini_ranger.cc:160] Time spent starting Ranger: real 8.613s user 0.003s sys 0.035s
Sep 27, 2026 7:03:05 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 7:03:05 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 7:03:05 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 7:03:06 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 7:03:06 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 07:03:07.529479 13799 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.121.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.121.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/rangerkms_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.121.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/rangerkms_127.13.121.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.121.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.121.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.121.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/HTTP_127.13.121.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.121.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/HTTP_127.13.121.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 07:03:07 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492587, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 07:03:07.705909 13799 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 07:03:07.725425 13799 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 07:03:08.705734 13799 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 07:03:08.705843 13799 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-kms
I20260927 07:03:08.705874 13799 mini_ranger_kms.cc:227] Using postgres at 127.13.121.212:43745
2026-09-27 07:03:09,038 [I] DB FLAVOR :POSTGRES
2026-09-27 07:03:09,038 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 07:03:09,039 [I] Checking connection
2026-09-27 07:03:09,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 07:03:09,301 [I] connection success
2026-09-27 07:03:09,301 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 07:03:09,301 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 07:03:09,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 07:03:09,547 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 07:03:09,547 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 07:03:09,548 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 07:03:09,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.121.212:43745/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 07:03:09,819 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 07:03:10.338589 13799 mini_ranger_kms.cc:310] Created kms service
I20260927 07:03:10.510314 13799 mini_ranger_kms.cc:326] Created kudu user
I20260927 07:03:10.557891 13799 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 07:03:10 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492587, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:11.009671 13799 mini_ranger_kms.cc:384] Added ranger policy
I20260927 07:03:11.010037 13799 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-kms:/tmp/dist-test-task8NDO8F/build/asan/bin/postgresql.jar:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task8NDO8F/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 07:03:11.010104 13799 mini_ranger_kms.cc:240] Using host: 127.13.121.212
I20260927 07:03:11.023699 13799 mini_ranger_kms.cc:276] Ranger KMS PID: 26021
I20260927 07:03:11.023837 13799 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.121.212:38215
Sep 27, 2026 7:03:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 7:03:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 7:03:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 7:03:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 7:03:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 7:03:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 07:03:13.083806 13799 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.378s user 0.086s sys 0.258s
I20260927 07:03:13.084081 13799 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 07:03:13.084266 13799 mini_ranger_kms.cc:401] 127.13.121.212:38215/kms/v1/keys
Sep 27 07:03:13 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790492593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.121.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 07:03:13 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.121.212@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
Sep 27 07:03:13 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.121.212@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
Sep 27 07:03:16 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:03:16.275514 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41419
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41419
--ranger_config_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38215/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:16.605947 26071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:16.606254 26071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:16.606384 26071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:16.613897 26071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:16.614027 26071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:16.614079 26071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:16.614156 26071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:16.614216 26071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:16.624555 26071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.121.212:38215/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41419
--ranger_config_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:16.625916 26071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:16.627353 26071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:16.639372 26079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:16.639402 26077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:16.639463 26076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:16.640152 26071 server_base.cc:1034] running on GCE node
I20260927 07:03:16.640977 26071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:16.642604 26071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:16.643805 26071 hybrid_clock.cc:648] HybridClock initialized: now 1790492596643769 us; error 30 us; skew 500 ppm
I20260927 07:03:16.644160 26071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:16 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492596, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:16.651082 26071 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:03:16.652704 26071 webserver.cc:460] Webserver started at http://127.13.121.254:43463/ using document root <none> and password file <none>
I20260927 07:03:16.653349 26071 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:16.653468 26071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:16.653761 26071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:03:16 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492596, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:16.712423 26071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "8275dde0af624d7884d2578b636ba491"
format_stamp: "Formatted at 2026-09-27 07:03:16 on dist-test-slave-p6hz"
server_key: "0372b0865f7512fe916ec0b9e53876f5"
server_key_iv: "cec551d6eb4811783453819b3253a989"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:16.713582 26071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "8275dde0af624d7884d2578b636ba491"
format_stamp: "Formatted at 2026-09-27 07:03:16 on dist-test-slave-p6hz"
server_key: "0372b0865f7512fe916ec0b9e53876f5"
server_key_iv: "cec551d6eb4811783453819b3253a989"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:16.720911 26071 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.007s
I20260927 07:03:16.739269 26086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:16.740792 26071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:03:16.741071 26071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "8275dde0af624d7884d2578b636ba491"
format_stamp: "Formatted at 2026-09-27 07:03:16 on dist-test-slave-p6hz"
server_key: "0372b0865f7512fe916ec0b9e53876f5"
server_key_iv: "cec551d6eb4811783453819b3253a989"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:16.741313 26071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:16.759522 26071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:16.763581 26071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:16.763883 26071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:16.782881 26071 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:41419
I20260927 07:03:16.782884 26137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:41419 every 8 connection(s)
I20260927 07:03:16.784335 26071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:03:16.788230 26138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:16.790165 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 26071
I20260927 07:03:16.790367 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:03:16.791193 13799 external_mini_cluster.cc:1428] Setting key 29589aac755f38d4bb44ea93cf125cdf
I20260927 07:03:16.802847 26138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Bootstrap starting.
I20260927 07:03:16.807744 26138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:16.809263 26138 log.cc:824] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:16.813090 26138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: No bootstrap required, opened a new log
Sep 27 07:03:16 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:16.821755 26138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } }
I20260927 07:03:16.822232 26138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:16.822335 26138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8275dde0af624d7884d2578b636ba491, State: Initialized, Role: FOLLOWER
I20260927 07:03:16.823050 26138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } }
I20260927 07:03:16.823364 26138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:03:16.823510 26138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:03:16.823699 26138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:16.826191 26138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } }
I20260927 07:03:16.826714 26138 leader_election.cc:302] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8275dde0af624d7884d2578b636ba491; no voters:
I20260927 07:03:16.827400 26138 leader_election.cc:288] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:03:16.827639 26143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:03:16.832638 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:16.793411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.13.121.254:41419)
0927 07:03:16.808118 (+ 14707us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:16.808139 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:16.808199 (+ 60us) server_negotiation.cc:407] Connection header received
0927 07:03:16.810900 (+ 2701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:16.810931 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:16.811471 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:16.811927 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:16.813480 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:16.815546 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:16.817324 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:16.818442 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:16.818621 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:16.823270 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:16.823280 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:16.823300 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:16.823367 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:16.827044 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:16.827969 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:16.827978 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:16.827987 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:16.828164 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:16.828583 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:16.828587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:16.828591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:16.829322 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:16.829606 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:16.830835 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0927 07:03:16.831291 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14426,"thread_start_us":14692,"threads_started":1}
I20260927 07:03:16.834973 26143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 1 LEADER]: Becoming Leader. State: Replica: 8275dde0af624d7884d2578b636ba491, State: Running, Role: LEADER
I20260927 07:03:16.835682 26143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } }
I20260927 07:03:16.836038 26138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:16.838841 26144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8275dde0af624d7884d2578b636ba491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } } }
I20260927 07:03:16.839224 26144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:16.839495 26145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8275dde0af624d7884d2578b636ba491. Latest consensus state: current_term: 1 leader_uuid: "8275dde0af624d7884d2578b636ba491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } } }
I20260927 07:03:16.839829 26145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:16.841144 26149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:03:16.846799 26149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:03:16.869513 26149 catalog_manager.cc:1347] Generated new cluster ID: d2ef8e4702064c5793bf9de48b60ea0f
I20260927 07:03:16.869648 26149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:03:16.884255 26138 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 07:03:16.885433 26149 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:03:16.887231 26149 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:03:16.905488 26149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Generated new TSK 0
I20260927 07:03:16.906870 26149 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:03:17.780493 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41419
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:03:18 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790492598, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 07:03:18.200344 26183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:18.200781 26183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:18.200999 26183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:18.212976 26183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:18.213141 26183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:18.213599 26183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:03:18.230103 26183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.121.212:38215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:18.232115 26183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:18.233852 26183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:18.263533 26193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:18.264331 26194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:18.273985 26196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:18.274389 26183 server_base.cc:1034] running on GCE node
I20260927 07:03:18.275092 26183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:18.276265 26183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:18.277493 26183 hybrid_clock.cc:648] HybridClock initialized: now 1790492598277437 us; error 53 us; skew 500 ppm
I20260927 07:03:18.277813 26183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:18 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492598, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:18.281069 26183 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:03:18.282794 26183 webserver.cc:460] Webserver started at http://127.13.121.193:32797/ using document root <none> and password file <none>
I20260927 07:03:18.283569 26183 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:18.283689 26183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:18.283978 26183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:03:18 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492598, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:18.303097 26183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "e3753f0db1c54cabb581896a612a332a"
format_stamp: "Formatted at 2026-09-27 07:03:18 on dist-test-slave-p6hz"
server_key: "f03d72785e2a6942b8062b18573fc6b7"
server_key_iv: "5a7171daf0cdad239b14ab4f98ce361e"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:18.304455 26183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "e3753f0db1c54cabb581896a612a332a"
format_stamp: "Formatted at 2026-09-27 07:03:18 on dist-test-slave-p6hz"
server_key: "f03d72785e2a6942b8062b18573fc6b7"
server_key_iv: "5a7171daf0cdad239b14ab4f98ce361e"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:18.322587 26183 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.000s sys 0.016s
I20260927 07:03:18.362823 26203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:18.366091 26183 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.007s sys 0.005s
I20260927 07:03:18.366408 26183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "e3753f0db1c54cabb581896a612a332a"
format_stamp: "Formatted at 2026-09-27 07:03:18 on dist-test-slave-p6hz"
server_key: "f03d72785e2a6942b8062b18573fc6b7"
server_key_iv: "5a7171daf0cdad239b14ab4f98ce361e"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:18.366992 26183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:18.460960 26183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:18.466784 26183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:18.467226 26183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:18.475231 26183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:18.477262 26183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:18.477416 26183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:18.477594 26183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:18.477691 26183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:03:18 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492598, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:18.746536 26183 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:43295
I20260927 07:03:18.746564 26315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:43295 every 8 connection(s)
I20260927 07:03:18.748410 26183 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:03:18.759552 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 26183
I20260927 07:03:18.759738 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:03:18.760324 13799 external_mini_cluster.cc:1428] Setting key da17585274004368922c01327d15ec9d
Sep 27 07:03:18 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492598, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:18.773928 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:18.751189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:55363 (local address 127.13.121.254:41419)
0927 07:03:18.755313 (+ 4124us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:18.755319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:18.755345 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:03:18.755416 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:18.755422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:18.755519 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:18.755641 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:18.757626 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:18.758805 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:18.761290 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:18.761977 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:18.762105 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:18.766673 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:18.766679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:18.766684 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:18.766725 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:18.769934 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:18.770951 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:18.770956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:18.770959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:18.771119 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:18.771704 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:18.771710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:18.771713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:18.772053 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:18.772185 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:18.773216 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0927 07:03:18.773464 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":236,"threads_started":1}
I20260927 07:03:18.774938 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:18.751752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41419 (local address 127.13.121.193:55363)
0927 07:03:18.752606 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:18.752660 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:18.754969 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:18.755929 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:18.755942 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:18.756587 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:18.757382 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:18.757409 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:18.758988 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:18.758994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:18.761099 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:18.761114 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:18.762445 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:18.762455 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:18.762776 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:18.763659 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:18.763690 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:18.766361 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:18.770125 (+ 3764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:18.770135 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:18.770156 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:18.770730 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:18.771267 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:18.771272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:18.771275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:18.771572 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:18.772253 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:18.772262 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:18.772747 (+ 485us) client_negotiation.cc:769] Sending connection context
0927 07:03:18.773057 (+ 310us) client_negotiation.cc:240] Negotiation successful
0927 07:03:18.773594 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":301,"threads_started":1}
I20260927 07:03:18.776806 26316 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41419
I20260927 07:03:18.777113 26316 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:18.777932 26316 heartbeater.cc:511] Master 127.13.121.254:41419 requested a full tablet report, sending...
I20260927 07:03:18.780459 26103 ts_manager.cc:194] Registered new tserver with Master: e3753f0db1c54cabb581896a612a332a (127.13.121.193:43295)
I20260927 07:03:18.782536 26103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:55363
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:03:18.828408 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41419
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:03:18 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492598, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
W20260927 07:03:19.274978 26322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:19.275662 26322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:19.275871 26322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:19.287688 26322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:19.287914 26322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:19.288115 26322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:03:19.303617 26322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.121.212:38215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:19.305575 26322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:19.307374 26322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:19.324244 26337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:19.323813 26336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:19.332778 26339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:19.333788 26322 server_base.cc:1034] running on GCE node
I20260927 07:03:19.334656 26322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:19.336076 26322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:19.337368 26322 hybrid_clock.cc:648] HybridClock initialized: now 1790492599337302 us; error 58 us; skew 500 ppm
I20260927 07:03:19.337780 26322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:19 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492599, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:19.341850 26322 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:03:19.351591 26322 webserver.cc:460] Webserver started at http://127.13.121.194:43397/ using document root <none> and password file <none>
I20260927 07:03:19.352625 26322 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:19.352774 26322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:19.353212 26322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:03:19 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492599, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:19.372556 26322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "037b7b469db043cb83a00c4db7e57626"
format_stamp: "Formatted at 2026-09-27 07:03:19 on dist-test-slave-p6hz"
server_key: "3bce421747602afc854963286ce257d6"
server_key_iv: "89f70dadc6edd08d209972da83144552"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:19.373706 26322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "037b7b469db043cb83a00c4db7e57626"
format_stamp: "Formatted at 2026-09-27 07:03:19 on dist-test-slave-p6hz"
server_key: "3bce421747602afc854963286ce257d6"
server_key_iv: "89f70dadc6edd08d209972da83144552"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:19.395643 26322 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.005s sys 0.013s
I20260927 07:03:19.422091 26346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:19.424362 26322 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.004s
I20260927 07:03:19.424737 26322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "037b7b469db043cb83a00c4db7e57626"
format_stamp: "Formatted at 2026-09-27 07:03:19 on dist-test-slave-p6hz"
server_key: "3bce421747602afc854963286ce257d6"
server_key_iv: "89f70dadc6edd08d209972da83144552"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:19.425119 26322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:19.485839 26322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:19.491284 26322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:19.491753 26322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:19.499516 26322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:19.507929 26322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:19.508124 26322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260927 07:03:19.508395 26322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:19.508488 26322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:19.623512 26322 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:39797
I20260927 07:03:19.623571 26458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:39797 every 8 connection(s)
I20260927 07:03:19.625116 26322 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:03:19.630559 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 26322
I20260927 07:03:19.630751 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:03:19.631371 13799 external_mini_cluster.cc:1428] Setting key 11e4683d6d4a00d6af63490246c87dfc
Sep 27 07:03:19 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492599, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:19.664742 26462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:19.628738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41419 (local address 127.13.121.194:34913)
0927 07:03:19.640052 (+ 11314us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:19.640116 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:19.644170 (+ 4054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:19.645282 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:19.645296 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:19.645983 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:19.646819 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:19.646851 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:19.648488 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:19.648493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:19.650868 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:19.650884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:19.651962 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:19.651969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:19.652264 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:19.653041 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:19.653071 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:19.655877 (+ 2806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:19.659840 (+ 3963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:19.659850 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:19.659870 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:19.660492 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:19.660950 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:19.660955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:19.660958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:19.661229 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:19.661936 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:19.661945 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:19.662438 (+ 493us) client_negotiation.cc:769] Sending connection context
0927 07:03:19.662750 (+ 312us) client_negotiation.cc:240] Negotiation successful
0927 07:03:19.663327 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10950,"thread_start_us":12136,"threads_started":1}
I20260927 07:03:19.668058 26461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:19.628046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:34913 (local address 127.13.121.254:41419)
0927 07:03:19.644655 (+ 16609us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:19.644661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:19.644685 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:03:19.644753 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:19.644759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:19.644854 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:19.644973 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:19.647206 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:19.648294 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:19.651085 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:19.651716 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:19.652686 (+ 970us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:19.656182 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:19.656189 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:19.656194 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:19.656240 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:19.659655 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:19.660667 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:19.660672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:19.660676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:19.660819 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:19.661359 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:19.661364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:19.661367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:19.661725 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:19.661852 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:19.667398 (+ 5546us) server_negotiation.cc:299] Negotiation successful
0927 07:03:19.667673 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16460,"thread_start_us":17013,"threads_started":1}
I20260927 07:03:19.669652 26459 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41419
I20260927 07:03:19.669977 26459 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:19.670833 26459 heartbeater.cc:511] Master 127.13.121.254:41419 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:03:19.672674 26103 ts_manager.cc:194] Registered new tserver with Master: 037b7b469db043cb83a00c4db7e57626 (127.13.121.194:39797)
I20260927 07:03:19.673827 26103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:34913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:03:19.712359 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41419
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:03:19.786824 26316 heartbeater.cc:503] Master 127.13.121.254:41419 was elected leader, sending a full tablet report...
W20260927 07:03:20.044145 26465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:20.044430 26465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:20.044575 26465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:20.051681 26465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:20.051775 26465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:20.051908 26465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:03:20.062346 26465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.121.212:38215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:20.063946 26465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:20.065248 26465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:20.078449 26471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:20.078522 26473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:20.078501 26470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:20.079234 26465 server_base.cc:1034] running on GCE node
I20260927 07:03:20.079958 26465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:20.081012 26465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:20.082242 26465 hybrid_clock.cc:648] HybridClock initialized: now 1790492600082189 us; error 48 us; skew 500 ppm
I20260927 07:03:20.082607 26465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:20 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492600, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:20.085899 26465 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:03:20.087620 26465 webserver.cc:460] Webserver started at http://127.13.121.195:41599/ using document root <none> and password file <none>
I20260927 07:03:20.088330 26465 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:20.088445 26465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:20.088762 26465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 07:03:20 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492600, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:20.107097 26465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "bf7499ebc8ee46babe4ef86c6e1b9457"
format_stamp: "Formatted at 2026-09-27 07:03:20 on dist-test-slave-p6hz"
server_key: "708f636e824b21e7d2f9ec6695595a8d"
server_key_iv: "69384c9f2af5f3dd97037cc71b153a0e"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:20.108006 26465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "bf7499ebc8ee46babe4ef86c6e1b9457"
format_stamp: "Formatted at 2026-09-27 07:03:20 on dist-test-slave-p6hz"
server_key: "708f636e824b21e7d2f9ec6695595a8d"
server_key_iv: "69384c9f2af5f3dd97037cc71b153a0e"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:20.113663 26465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:03:20.131565 26480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:20.133221 26465 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 07:03:20.133433 26465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "bf7499ebc8ee46babe4ef86c6e1b9457"
format_stamp: "Formatted at 2026-09-27 07:03:20 on dist-test-slave-p6hz"
server_key: "708f636e824b21e7d2f9ec6695595a8d"
server_key_iv: "69384c9f2af5f3dd97037cc71b153a0e"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:20.133666 26465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:20.167598 26465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:20.171609 26465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:20.171888 26465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:20.172833 26465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:20.174280 26465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:20.174362 26465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:20.174480 26465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:20.174531 26465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:20.204759 26465 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:38611
I20260927 07:03:20.204793 26592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:38611 every 8 connection(s)
I20260927 07:03:20.206013 26465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:03:20.207283 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 26465
I20260927 07:03:20.207453 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:03:20.207971 13799 external_mini_cluster.cc:1428] Setting key 5aa54944a8610bcdf8d3c64cbf7370a7
Sep 27 07:03:20 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492600, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:20.226266 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:20.208576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:47885 (local address 127.13.121.254:41419)
0927 07:03:20.209064 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:20.209070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:20.210147 (+ 1077us) server_negotiation.cc:407] Connection header received
0927 07:03:20.212146 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:20.212150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:20.212226 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:20.212311 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:20.213690 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.214439 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.216207 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.216707 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.216812 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:20.220241 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:20.220248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:20.220251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:20.220290 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:20.223086 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:20.224081 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:20.224085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:20.224087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:20.224226 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:20.224659 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:20.224663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:20.224664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:20.224940 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:20.225070 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:20.225806 (+ 736us) server_negotiation.cc:299] Negotiation successful
0927 07:03:20.225990 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":476,"threads_started":1}
I20260927 07:03:20.227248 26596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:20.209168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41419 (local address 127.13.121.195:47885)
0927 07:03:20.209983 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:20.210032 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:20.211898 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:20.212505 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:20.212513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:20.212945 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:20.213513 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:20.213531 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.214617 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.214621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:20.216034 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:20.216048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.216949 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.216955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:20.217210 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:20.217883 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:20.217908 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:20.219972 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:20.223269 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:20.223279 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:20.223299 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:20.223925 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:20.224332 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:20.224336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:20.224338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:20.224565 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:20.225073 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:20.225079 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:20.225460 (+ 381us) client_negotiation.cc:769] Sending connection context
0927 07:03:20.225721 (+ 261us) client_negotiation.cc:240] Negotiation successful
0927 07:03:20.226124 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":301,"threads_started":1}
I20260927 07:03:20.228593 26593 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41419
I20260927 07:03:20.228874 26593 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:20.229560 26593 heartbeater.cc:511] Master 127.13.121.254:41419 requested a full tablet report, sending...
I20260927 07:03:20.231015 26103 ts_manager.cc:194] Registered new tserver with Master: bf7499ebc8ee46babe4ef86c6e1b9457 (127.13.121.195:38611)
I20260927 07:03:20.232015 26103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:47885
I20260927 07:03:20.236632 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:03:20.238379 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 26183
Sep 27 07:03:20 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:03:20.255416 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:20.239310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48912 (local address 127.13.121.193:43295)
0927 07:03:20.239946 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:20.239953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:20.239978 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:03:20.240033 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:20.240039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:20.240242 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:20.240382 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:20.241752 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.242814 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.244381 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.245006 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.245130 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:20.249470 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:20.249485 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:20.249502 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:20.249543 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:20.252461 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:20.253302 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:20.253310 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:20.253312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:20.253474 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:20.253883 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:20.253887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:20.253889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:20.254313 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:20.254555 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:20.254843 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 07:03:20.255099 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":299,"threads_started":1}
I20260927 07:03:20.256776 26311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:20.340286 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 26183
I20260927 07:03:20.353214 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 26322
Sep 27 07:03:20 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:03:20.369594 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:20.354216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.13.121.194:39797)
0927 07:03:20.354735 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:20.354742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:20.354767 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:03:20.354826 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:20.354832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:20.355044 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:20.355206 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:20.356403 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.357425 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.358824 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.359507 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.359606 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:20.363814 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:20.363827 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:20.363843 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:20.363912 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:20.366621 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:20.367524 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:20.367532 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:20.367534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:20.367678 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:20.367986 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:20.367989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:20.367991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:20.368336 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:20.368581 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:20.368976 (+ 395us) server_negotiation.cc:299] Negotiation successful
0927 07:03:20.369255 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":246,"threads_started":1}
I20260927 07:03:20.370926 26454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:20.453197 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 26322
I20260927 07:03:20.466214 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 26465
Sep 27 07:03:20 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:03:20.483239 26602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:20.467227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.13.121.195:38611)
0927 07:03:20.467892 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:20.467899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:20.467931 (+ 32us) server_negotiation.cc:407] Connection header received
0927 07:03:20.467992 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:20.467999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:20.468210 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:20.468389 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:20.469749 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.470788 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.472295 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:20.472890 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:20.473029 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:20.477671 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:20.477686 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:20.477702 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:20.477739 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:20.480510 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:20.481219 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:20.481226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:20.481229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:20.481359 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:20.481705 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:20.481708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:20.481710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:20.482057 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:20.482360 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:20.482627 (+ 267us) server_negotiation.cc:299] Negotiation successful
0927 07:03:20.482880 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":333,"threads_started":1}
I20260927 07:03:20.484491 26588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:20.566668 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 26465
I20260927 07:03:20.579603 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 26071
I20260927 07:03:20.580812 26133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:20.684701 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 26071
I20260927 07:03:20.706027 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41419
--webserver_interface=127.13.121.254
--webserver_port=43463
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41419
--ranger_config_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38215/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.121.212:38215/kms with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:21.023800 26605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:21.024103 26605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:21.024205 26605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:21.031581 26605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:21.031704 26605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:21.031755 26605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:21.031809 26605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:21.031855 26605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:21.042196 26605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.121.212:38215/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41419
--ranger_config_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=43463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:21.043514 26605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:21.044772 26605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:21.056142 26610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:21.056190 26613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:21.056241 26611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:21.056869 26605 server_base.cc:1034] running on GCE node
I20260927 07:03:21.057677 26605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:21.059127 26605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:21.060371 26605 hybrid_clock.cc:648] HybridClock initialized: now 1790492601060323 us; error 41 us; skew 500 ppm
I20260927 07:03:21.060742 26605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:21 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492601, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:21.063855 26605 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:03:21.065317 26605 webserver.cc:460] Webserver started at http://127.13.121.254:43463/ using document root <none> and password file <none>
I20260927 07:03:21.065985 26605 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:21.066108 26605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 07:03:21 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492601, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:21.130569 26605 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:03:21.134431 26621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:21.136148 26605 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 07:03:21.136368 26605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "8275dde0af624d7884d2578b636ba491"
format_stamp: "Formatted at 2026-09-27 07:03:16 on dist-test-slave-p6hz"
server_key: "0372b0865f7512fe916ec0b9e53876f5"
server_key_iv: "cec551d6eb4811783453819b3253a989"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:21.137451 26605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:21.187445 26605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:21.191772 26605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:21.192026 26605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:21.209741 26605 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:41419
I20260927 07:03:21.209765 26672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:41419 every 8 connection(s)
I20260927 07:03:21.211277 26605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:03:21.218278 26673 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:03:21.218967 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 26605
I20260927 07:03:21.220019 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:43295
--local_ip_for_outbound_sockets=127.13.121.193
--tserver_master_addrs=127.13.121.254:41419
--webserver_port=32797
--webserver_interface=127.13.121.193
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.121.212:38215/kms with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:03:21.223315 26673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Bootstrap starting.
I20260927 07:03:21.249356 26673 log.cc:824] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:21.261829 26673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:03:21.262614 26673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Bootstrap complete.
I20260927 07:03:21.275993 26673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } }
I20260927 07:03:21.277566 26673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8275dde0af624d7884d2578b636ba491, State: Initialized, Role: FOLLOWER
I20260927 07:03:21.278509 26673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } }
I20260927 07:03:21.278851 26673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:03:21.278987 26673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:03:21.279202 26673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:03:21.285251 26673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } }
I20260927 07:03:21.285972 26673 leader_election.cc:302] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8275dde0af624d7884d2578b636ba491; no voters:
I20260927 07:03:21.286691 26673 leader_election.cc:288] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 07:03:21.286840 26677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 07:03:21.287978 26677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [term 2 LEADER]: Becoming Leader. State: Replica: 8275dde0af624d7884d2578b636ba491, State: Running, Role: LEADER
I20260927 07:03:21.288538 26677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } }
I20260927 07:03:21.289259 26673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:21.291196 26679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8275dde0af624d7884d2578b636ba491. Latest consensus state: current_term: 2 leader_uuid: "8275dde0af624d7884d2578b636ba491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } } }
I20260927 07:03:21.291571 26679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:21.292079 26678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8275dde0af624d7884d2578b636ba491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8275dde0af624d7884d2578b636ba491" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41419 } } }
I20260927 07:03:21.292369 26678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491 [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:21.295854 26683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:03:21.301874 26683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:03:21.303646 26683 catalog_manager.cc:1259] Loaded cluster ID: d2ef8e4702064c5793bf9de48b60ea0f
I20260927 07:03:21.303844 26683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:03:21.307976 26683 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:03:21.309582 26683 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Loaded TSK: 0
I20260927 07:03:21.311470 26683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:03:21.314595 26673 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260927 07:03:21.749181 26675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:21.749701 26675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:21.749894 26675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:21.762018 26675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:21.762243 26675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:21.762476 26675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:03:21.778496 26675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.121.212:38215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:43295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=32797
--tserver_master_addrs=127.13.121.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:21.780467 26675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:21.782241 26675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:21.814405 26711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:21.815856 26712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:21.828890 26714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:21.829851 26675 server_base.cc:1034] running on GCE node
I20260927 07:03:21.830806 26675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:21.832317 26675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:21.833812 26675 hybrid_clock.cc:648] HybridClock initialized: now 1790492601833732 us; error 68 us; skew 500 ppm
I20260927 07:03:21.834240 26675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:21 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492601, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:21.842648 26675 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:03:21.855646 26675 webserver.cc:460] Webserver started at http://127.13.121.193:32797/ using document root <none> and password file <none>
I20260927 07:03:21.856907 26675 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:21.857128 26675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 07:03:21 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492601, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:21.892163 26675 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.002s
I20260927 07:03:21.901561 26723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:21.903990 26675 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.010s sys 0.000s
I20260927 07:03:21.904289 26675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "e3753f0db1c54cabb581896a612a332a"
format_stamp: "Formatted at 2026-09-27 07:03:18 on dist-test-slave-p6hz"
server_key: "f03d72785e2a6942b8062b18573fc6b7"
server_key_iv: "5a7171daf0cdad239b14ab4f98ce361e"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:21.905514 26675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:21.934417 26675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:21.940194 26675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:21.940548 26675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:21.941910 26675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:21.943804 26675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:21.943917 26675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:21.944032 26675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:21.944103 26675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:21.998626 26675 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:43295
I20260927 07:03:21.998636 26835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:43295 every 8 connection(s)
I20260927 07:03:22.000676 26675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:03:22.011099 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 26675
I20260927 07:03:22.011943 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:39797
--local_ip_for_outbound_sockets=127.13.121.194
--tserver_master_addrs=127.13.121.254:41419
--webserver_port=43397
--webserver_interface=127.13.121.194
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.121.212:38215/kms with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:03:22 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492601, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:22.031737 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:22.004112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41419 (local address 127.13.121.193:47279)
0927 07:03:22.005375 (+ 1263us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:22.005424 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:22.007891 (+ 2467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:22.009693 (+ 1802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:22.009705 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:22.010362 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:22.011224 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:22.011254 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:22.013774 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:22.013780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:22.015958 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:22.015994 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:22.018061 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:22.018070 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:22.018359 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:22.019426 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:22.019466 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:22.022260 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:22.026949 (+ 4689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:22.026961 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:22.026978 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:22.027531 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:22.028044 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:22.028056 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:22.028058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:22.028244 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:22.029086 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:22.029095 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:22.029438 (+ 343us) client_negotiation.cc:769] Sending connection context
0927 07:03:22.029692 (+ 254us) client_negotiation.cc:240] Negotiation successful
0927 07:03:22.030312 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"thread_start_us":748,"threads_started":1}
I20260927 07:03:22.031731 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:22.004094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:47279 (local address 127.13.121.254:41419)
0927 07:03:22.006416 (+ 2322us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:22.006432 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:22.006474 (+ 42us) server_negotiation.cc:407] Connection header received
0927 07:03:22.008674 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:22.008691 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:22.009104 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:22.009429 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:22.011542 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:22.013513 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:22.016215 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:22.017387 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:22.017621 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:22.022680 (+ 5059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:22.022694 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:22.022717 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:22.022778 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:22.026748 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:22.027715 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:22.027724 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:22.027731 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:22.027912 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:22.028373 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:22.028379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:22.028383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:22.028931 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:22.029132 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:22.029961 (+ 829us) server_negotiation.cc:299] Negotiation successful
0927 07:03:22.030341 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2087,"thread_start_us":1883,"threads_started":1}
I20260927 07:03:22.034879 26836 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41419
I20260927 07:03:22.035246 26836 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:22.036033 26836 heartbeater.cc:511] Master 127.13.121.254:41419 requested a full tablet report, sending...
W20260927 07:03:22.038928 26836 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:41419 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 07:03:22.039326 26836 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:22.039665 26836 heartbeater.cc:511] Master 127.13.121.254:41419 requested a full tablet report, sending...
I20260927 07:03:22.041020 26836 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:22.041296 26836 heartbeater.cc:511] Master 127.13.121.254:41419 requested a full tablet report, sending...
W20260927 07:03:22.042505 26836 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 07:03:22.453524 26840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:22.453902 26840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:22.454061 26840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:22.462065 26840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:22.462208 26840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:22.462360 26840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:03:22.478288 26840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.121.212:38215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:39797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=43397
--tserver_master_addrs=127.13.121.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:22.480268 26840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:22.482091 26840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:22.499280 26854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:22.499301 26855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:22.500756 26858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:22.501435 26840 server_base.cc:1034] running on GCE node
I20260927 07:03:22.502111 26840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:22.503500 26840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:22.504760 26840 hybrid_clock.cc:648] HybridClock initialized: now 1790492602504678 us; error 63 us; skew 500 ppm
I20260927 07:03:22.505229 26840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:22 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492602, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:22.509829 26840 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:03:22.533355 26840 webserver.cc:460] Webserver started at http://127.13.121.194:43397/ using document root <none> and password file <none>
I20260927 07:03:22.534531 26840 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:22.534755 26840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 07:03:22 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492602, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:22.576563 26840 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.004s sys 0.012s
I20260927 07:03:22.580439 26867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:22.582226 26840 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 07:03:22.582435 26840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "037b7b469db043cb83a00c4db7e57626"
format_stamp: "Formatted at 2026-09-27 07:03:19 on dist-test-slave-p6hz"
server_key: "3bce421747602afc854963286ce257d6"
server_key_iv: "89f70dadc6edd08d209972da83144552"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:22.583922 26840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 27 07:03:22 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790492602, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:22.601903 26840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:22.607020 26840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:22.607537 26840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:22.611569 26840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:22.619975 26840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:22.620186 26840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:22.620354 26840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:22.620468 26840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:22.675693 26840 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:39797
I20260927 07:03:22.675734 26980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:39797 every 8 connection(s)
I20260927 07:03:22.677191 26840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:03:22.681993 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 26840
I20260927 07:03:22.682850 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.121.212:38215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:38611
--local_ip_for_outbound_sockets=127.13.121.195
--tserver_master_addrs=127.13.121.254:41419
--webserver_port=41599
--webserver_interface=127.13.121.195
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.121.212:38215/kms with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:03:22 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492602, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:22.718626 26983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:22.687612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:43159 (local address 127.13.121.254:41419)
0927 07:03:22.693100 (+ 5488us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:22.693107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:22.695240 (+ 2133us) server_negotiation.cc:407] Connection header received
0927 07:03:22.696990 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:22.696996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:22.697091 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:22.697191 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:22.699502 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:22.700605 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:22.703812 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:22.704453 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:22.704558 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:22.710142 (+ 5584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:22.710151 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:22.710156 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:22.710207 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:22.714100 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:22.715343 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:22.715350 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:22.715353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:22.715543 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:22.716169 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:22.716185 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:22.716188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:22.716788 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:22.716937 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:22.717945 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0927 07:03:22.718189 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5334,"thread_start_us":5199,"threads_started":1}
I20260927 07:03:22.721148 26982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:22.686928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41419 (local address 127.13.121.194:43159)
0927 07:03:22.694082 (+ 7154us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:22.694132 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:22.696484 (+ 2352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:22.697703 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:22.697716 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:22.698366 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:22.699245 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:22.699275 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:22.700826 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:22.700832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:22.703596 (+ 2764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:22.703618 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:22.704939 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:22.704946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:22.705220 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:22.706689 (+ 1469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:22.706721 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:22.709583 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:22.714352 (+ 4769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:22.714365 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:22.714402 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:22.715121 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:22.715720 (+ 599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:22.715725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:22.715729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:22.716022 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:22.717036 (+ 1014us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:22.717046 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:22.717524 (+ 478us) client_negotiation.cc:769] Sending connection context
0927 07:03:22.719200 (+ 1676us) client_negotiation.cc:240] Negotiation successful
0927 07:03:22.719822 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6842,"thread_start_us":6599,"threads_started":1}
I20260927 07:03:22.723512 26981 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41419
I20260927 07:03:22.723811 26981 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:22.724764 26981 heartbeater.cc:511] Master 127.13.121.254:41419 requested a full tablet report, sending...
I20260927 07:03:22.727033 26638 ts_manager.cc:194] Registered new tserver with Master: 037b7b469db043cb83a00c4db7e57626 (127.13.121.194:39797)
I20260927 07:03:22.729295 26638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:43159
Sep 27 07:03:23 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492602, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:23.047374 26836 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:23.047814 26836 heartbeater.cc:511] Master 127.13.121.254:41419 requested a full tablet report, sending...
I20260927 07:03:23.049028 26638 ts_manager.cc:194] Registered new tserver with Master: e3753f0db1c54cabb581896a612a332a (127.13.121.193:43295)
I20260927 07:03:23.050189 26638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:47279
W20260927 07:03:23.150380 26984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:23.150904 26984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:23.151108 26984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:23.163362 26984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:23.163655 26984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:23.163908 26984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:03:23.179911 26984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.121.212:38215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:38611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=41599
--tserver_master_addrs=127.13.121.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:23.181793 26984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:23.183665 26984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:23.222353 26991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:23.230388 26994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:23.231442 26990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:23.234365 26984 server_base.cc:1034] running on GCE node
I20260927 07:03:23.235446 26984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:23.242208 26984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:23.243638 26984 hybrid_clock.cc:648] HybridClock initialized: now 1790492603243564 us; error 79 us; skew 500 ppm
I20260927 07:03:23.244141 26984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:23 dist-test-slave-p6hz krb5kdc[25627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492603, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:23.248520 26984 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:03:23.254550 26984 webserver.cc:460] Webserver started at http://127.13.121.195:41599/ using document root <none> and password file <none>
I20260927 07:03:23.255365 26984 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:23.255543 26984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 07:03:23 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492603, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:23.288846 26984 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.000s sys 0.008s
I20260927 07:03:23.292210 27001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:23.293756 26984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:03:23.293965 26984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "bf7499ebc8ee46babe4ef86c6e1b9457"
format_stamp: "Formatted at 2026-09-27 07:03:20 on dist-test-slave-p6hz"
server_key: "708f636e824b21e7d2f9ec6695595a8d"
server_key_iv: "69384c9f2af5f3dd97037cc71b153a0e"
server_key_version: "kuduclusterkey@0"
I20260927 07:03:23.295003 26984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:23.334403 26984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:23.341542 26984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:23.341851 26984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:23.352308 26984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:23.354305 26984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:23.354421 26984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:23.354549 26984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:23.354620 26984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 07:03:23 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492602, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for HTTP/127.13.121.212@KRBTEST.COM
I20260927 07:03:23.504092 26984 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:38611
I20260927 07:03:23.504145 27114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:38611 every 8 connection(s)
I20260927 07:03:23.506032 26984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:03:23.513351 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 26984
Sep 27 07:03:23 dist-test-slave-p6hz krb5kdc[25627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492603, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:23.592730 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.514525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.13.121.254:41419)
0927 07:03:23.559538 (+ 45013us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:23.559544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:23.561542 (+ 1998us) server_negotiation.cc:407] Connection header received
0927 07:03:23.561637 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:23.561643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:23.561737 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:23.561828 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:23.569338 (+ 7510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.570512 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.575258 (+ 4746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.575835 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.575942 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.581187 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:23.581194 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:23.581198 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:23.581241 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:23.584657 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:23.586010 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:23.586016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:23.586020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:23.586169 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:23.586618 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:23.586623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:23.586627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:23.587072 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:23.587375 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:23.591928 (+ 4553us) server_negotiation.cc:299] Negotiation successful
0927 07:03:23.592346 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44871,"thread_start_us":51661,"threads_started":1}
I20260927 07:03:23.601218 27118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.528468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:38469 (local address 127.13.121.254:41419)
0927 07:03:23.566506 (+ 38038us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:23.566512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:23.566530 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:03:23.566590 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:23.566595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:23.566683 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:23.566793 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:23.573445 (+ 6652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.574518 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.580029 (+ 5511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.580722 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.580845 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.589938 (+ 9093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:23.589946 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:23.589950 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:23.589993 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:23.597208 (+ 7215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:23.598294 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:23.598300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:23.598303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:23.598451 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:23.599009 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:23.599014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:23.599017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:23.599406 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:23.599644 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:23.600551 (+ 907us) server_negotiation.cc:299] Negotiation successful
0927 07:03:23.600799 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37870,"thread_start_us":31952,"threads_started":1}
I20260927 07:03:23.603619 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.527795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41419 (local address 127.13.121.195:38469)
0927 07:03:23.557534 (+ 29739us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:23.557604 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:23.565378 (+ 7774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:23.567438 (+ 2060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:23.567453 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:23.568154 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:23.573210 (+ 5056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.573242 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.574728 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.574734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.579800 (+ 5066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.579825 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.582095 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.582138 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.584934 (+ 2796us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.585661 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:23.585692 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:23.589566 (+ 3874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:23.597413 (+ 7847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:23.597424 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:23.597446 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:23.598139 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:23.598625 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:23.598630 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:23.598634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:23.598879 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:23.599536 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:23.599545 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:23.600091 (+ 546us) client_negotiation.cc:769] Sending connection context
0927 07:03:23.601472 (+ 1381us) client_negotiation.cc:240] Negotiation successful
0927 07:03:23.602111 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29357,"thread_start_us":31619,"threads_started":1}
I20260927 07:03:23.605232 27115 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41419
I20260927 07:03:23.605581 27115 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:23.606410 27115 heartbeater.cc:511] Master 127.13.121.254:41419 requested a full tablet report, sending...
I20260927 07:03:23.608134 26637 ts_manager.cc:194] Registered new tserver with Master: bf7499ebc8ee46babe4ef86c6e1b9457 (127.13.121.195:38611)
I20260927 07:03:23.609102 26637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:38469
I20260927 07:03:23.621002 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:03:23.639187 27118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.624237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.13.121.254:41419)
0927 07:03:23.624433 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:23.624439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:23.624754 (+ 315us) server_negotiation.cc:407] Connection header received
0927 07:03:23.624914 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:23.624918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:23.625018 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:23.625100 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:23.627009 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.627786 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.629694 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.630150 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.630255 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.633522 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:23.633527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:23.633530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:23.633572 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:23.636394 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:23.637309 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:23.637314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:23.637316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:23.637500 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:23.637867 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:23.637870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:23.637872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:23.638177 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:23.638346 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:23.638723 (+ 377us) server_negotiation.cc:299] Negotiation successful
0927 07:03:23.638901 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 07:03:23.644467 26637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:03:23.651815 26637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:03:23.681348 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.666533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.0.0.1:48914)
0927 07:03:23.671627 (+ 5094us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:23.671678 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:23.671883 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:23.673200 (+ 1317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:23.673207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:23.673242 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:23.673611 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.673628 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.675655 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.675661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.678062 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.678081 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.680344 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.680351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.680575 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.680634 (+ 59us) client_negotiation.cc:769] Sending connection context
0927 07:03:23.680773 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 07:03:23.680979 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4812,"thread_start_us":5180,"threads_started":1}
I20260927 07:03:23.682123 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.667386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48914 (local address 127.13.121.193:43295)
0927 07:03:23.667963 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:23.667972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:23.672150 (+ 4178us) server_negotiation.cc:407] Connection header received
0927 07:03:23.672270 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:23.672277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:23.672543 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:23.672735 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:23.673811 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.675484 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.678277 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.679775 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.679975 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.680175 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:23.681609 (+ 1434us) server_negotiation.cc:299] Negotiation successful
0927 07:03:23.681797 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":675,"threads_started":1}
I20260927 07:03:23.686575 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.666623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:39797 (local address 127.0.0.1:40772)
0927 07:03:23.672838 (+ 6215us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:23.672854 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:23.673065 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:23.673980 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:23.673985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:23.674011 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:23.675465 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.675479 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.677434 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.677438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.680765 (+ 3327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.680780 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.685821 (+ 5041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.685828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.686006 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.686040 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 07:03:23.686126 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 07:03:23.686264 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6092,"thread_start_us":5971,"threads_started":1}
I20260927 07:03:23.686748 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.666830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.13.121.194:39797)
0927 07:03:23.667437 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:23.667446 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:23.672954 (+ 5508us) server_negotiation.cc:407] Connection header received
0927 07:03:23.673250 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:23.673258 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:23.673491 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:23.673694 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:23.675643 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.677222 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.680954 (+ 3732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.685301 (+ 4347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.685497 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.685662 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:23.686237 (+ 575us) server_negotiation.cc:299] Negotiation successful
0927 07:03:23.686416 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":296,"threads_started":1}
I20260927 07:03:23.686789 26771 tablet_service.cc:1467] Processing CreateTablet for tablet c7655708d0264fdda9fe0cd8eef71aa5 (DEFAULT_TABLE table=test-table [id=f412da62340f4f4ab37b66c75430323b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:23.688423 26771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7655708d0264fdda9fe0cd8eef71aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:23.690738 26915 tablet_service.cc:1467] Processing CreateTablet for tablet c7655708d0264fdda9fe0cd8eef71aa5 (DEFAULT_TABLE table=test-table [id=f412da62340f4f4ab37b66c75430323b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:23.691568 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.681886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:38611 (local address 127.0.0.1:58362)
0927 07:03:23.682595 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:23.682615 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:23.682717 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:23.685482 (+ 2765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:23.685487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:23.685514 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:23.685692 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.685702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.687881 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.687886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.689224 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.689237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.690804 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.690811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.690996 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.691031 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:03:23.691120 (+ 89us) client_negotiation.cc:240] Negotiation successful
0927 07:03:23.691290 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":3702,"threads_started":1}
I20260927 07:03:23.691921 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.673388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.13.121.195:38611)
0927 07:03:23.684862 (+ 11474us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:23.684870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:23.684904 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:03:23.685003 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:23.685011 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:23.685272 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:23.685850 (+ 578us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:23.686088 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.687758 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.689372 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.690650 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.690827 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.690990 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:23.691199 (+ 209us) server_negotiation.cc:299] Negotiation successful
0927 07:03:23.691355 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11298,"thread_start_us":11995,"threads_started":1}
I20260927 07:03:23.692266 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7655708d0264fdda9fe0cd8eef71aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:23.698814 27050 tablet_service.cc:1467] Processing CreateTablet for tablet c7655708d0264fdda9fe0cd8eef71aa5 (DEFAULT_TABLE table=test-table [id=f412da62340f4f4ab37b66c75430323b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:23.700193 27050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7655708d0264fdda9fe0cd8eef71aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:23.704479 27143 tablet_bootstrap.cc:492] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: Bootstrap starting.
I20260927 07:03:23.708290 27143 tablet_bootstrap.cc:654] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:23.709349 27144 tablet_bootstrap.cc:492] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: Bootstrap starting.
I20260927 07:03:23.709600 27143 log.cc:824] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:23.713131 27143 tablet_bootstrap.cc:492] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: No bootstrap required, opened a new log
I20260927 07:03:23.713140 27144 tablet_bootstrap.cc:654] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:23.713430 27143 ts_tablet_manager.cc:1397] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260927 07:03:23.714471 27144 log.cc:824] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:23.714700 27146 tablet_bootstrap.cc:492] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: Bootstrap starting.
I20260927 07:03:23.719110 27146 tablet_bootstrap.cc:654] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:23.720607 27146 log.cc:824] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:23.722558 27144 tablet_bootstrap.cc:492] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: No bootstrap required, opened a new log
I20260927 07:03:23.722462 27143 raft_consensus.cc:348] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } }
I20260927 07:03:23.722885 27144 ts_tablet_manager.cc:1397] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260927 07:03:23.722949 27143 raft_consensus.cc:374] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:23.723052 27143 raft_consensus.cc:729] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3753f0db1c54cabb581896a612a332a, State: Initialized, Role: FOLLOWER
I20260927 07:03:23.723899 27143 consensus_queue.cc:260] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } }
I20260927 07:03:23.724970 27146 tablet_bootstrap.cc:492] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: No bootstrap required, opened a new log
I20260927 07:03:23.725262 27146 ts_tablet_manager.cc:1397] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260927 07:03:23.732026 26836 heartbeater.cc:503] Master 127.13.121.254:41419 was elected leader, sending a full tablet report...
I20260927 07:03:23.732559 27143 ts_tablet_manager.cc:1428] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: Time spent starting tablet: real 0.019s user 0.013s sys 0.005s
I20260927 07:03:23.734157 26981 heartbeater.cc:503] Master 127.13.121.254:41419 was elected leader, sending a full tablet report...
I20260927 07:03:23.734427 27144 raft_consensus.cc:348] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } }
I20260927 07:03:23.734875 27144 raft_consensus.cc:374] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:23.735001 27144 raft_consensus.cc:729] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 037b7b469db043cb83a00c4db7e57626, State: Initialized, Role: FOLLOWER
I20260927 07:03:23.735472 27146 raft_consensus.cc:348] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } }
I20260927 07:03:23.735687 27144 consensus_queue.cc:260] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } }
I20260927 07:03:23.735898 27146 raft_consensus.cc:374] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:23.735989 27146 raft_consensus.cc:729] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7499ebc8ee46babe4ef86c6e1b9457, State: Initialized, Role: FOLLOWER
I20260927 07:03:23.736722 27146 consensus_queue.cc:260] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } }
I20260927 07:03:23.743978 27115 heartbeater.cc:503] Master 127.13.121.254:41419 was elected leader, sending a full tablet report...
I20260927 07:03:23.745042 27146 ts_tablet_manager.cc:1428] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: Time spent starting tablet: real 0.020s user 0.016s sys 0.003s
I20260927 07:03:23.745033 27144 ts_tablet_manager.cc:1428] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: Time spent starting tablet: real 0.022s user 0.017s sys 0.004s
W20260927 07:03:23.753683 26837 tablet.cc:2369] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:03:23.827227 27116 tablet.cc:2369] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:03:23.905378 27151 raft_consensus.cc:482] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:23.905687 27151 raft_consensus.cc:504] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } }
I20260927 07:03:23.907395 27151 leader_election.cc:288] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3753f0db1c54cabb581896a612a332a (127.13.121.193:43295), 037b7b469db043cb83a00c4db7e57626 (127.13.121.194:39797)
I20260927 07:03:23.913996 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.907699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:39771 (local address 127.13.121.193:43295)
0927 07:03:23.907908 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:23.907914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:23.908190 (+ 276us) server_negotiation.cc:407] Connection header received
0927 07:03:23.908273 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:23.908277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:23.908344 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:23.908431 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:23.908814 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.909772 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.912276 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.913178 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.913320 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.913392 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:23.913633 (+ 241us) server_negotiation.cc:299] Negotiation successful
0927 07:03:23.913757 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 07:03:23.914016 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.907622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.13.121.195:39771)
0927 07:03:23.907944 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:23.907964 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:23.908112 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:23.908467 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:23.908472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:23.908503 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:23.908672 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.908680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.909933 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.909938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.912079 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.912097 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.913319 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.913324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.913487 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.913511 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 07:03:23.913594 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 07:03:23.913735 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":59}
I20260927 07:03:23.914287 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.907948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:46659 (local address 127.13.121.194:39797)
0927 07:03:23.908129 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:23.908134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:23.908403 (+ 269us) server_negotiation.cc:407] Connection header received
0927 07:03:23.908578 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:23.908581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:23.908647 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:23.908713 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:23.909210 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.910118 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.912647 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.913423 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.913572 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.913642 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:23.913923 (+ 281us) server_negotiation.cc:299] Negotiation successful
0927 07:03:23.914044 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 07:03:23.914597 27152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.907668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:39797 (local address 127.13.121.195:46659)
0927 07:03:23.908299 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:23.908313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:23.908434 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:23.908750 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:23.908755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:23.908780 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:23.909069 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.909076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.910260 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.910268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.912495 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:23.912508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.913624 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.913630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:23.913784 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.913809 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 07:03:23.913884 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 07:03:23.914012 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":1,"thread_start_us":371,"threads_started":1}
I20260927 07:03:23.915066 26791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7655708d0264fdda9fe0cd8eef71aa5" candidate_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3753f0db1c54cabb581896a612a332a" is_pre_election: true
I20260927 07:03:23.915689 26791 raft_consensus.cc:2454] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf7499ebc8ee46babe4ef86c6e1b9457 in term 0.
I20260927 07:03:23.915570 26935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7655708d0264fdda9fe0cd8eef71aa5" candidate_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "037b7b469db043cb83a00c4db7e57626" is_pre_election: true
I20260927 07:03:23.916131 26935 raft_consensus.cc:2454] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf7499ebc8ee46babe4ef86c6e1b9457 in term 0.
I20260927 07:03:23.916417 27002 leader_election.cc:302] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf7499ebc8ee46babe4ef86c6e1b9457, e3753f0db1c54cabb581896a612a332a; no voters:
I20260927 07:03:23.916895 27151 raft_consensus.cc:2781] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:03:23.917059 27151 raft_consensus.cc:482] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:03:23.917157 27151 raft_consensus.cc:3037] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:23.919322 27151 raft_consensus.cc:504] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } }
I20260927 07:03:23.919934 27151 leader_election.cc:288] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [CANDIDATE]: Term 1 election: Requested vote from peers e3753f0db1c54cabb581896a612a332a (127.13.121.193:43295), 037b7b469db043cb83a00c4db7e57626 (127.13.121.194:39797)
I20260927 07:03:23.920415 26791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7655708d0264fdda9fe0cd8eef71aa5" candidate_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3753f0db1c54cabb581896a612a332a"
I20260927 07:03:23.920506 26935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7655708d0264fdda9fe0cd8eef71aa5" candidate_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "037b7b469db043cb83a00c4db7e57626"
I20260927 07:03:23.920607 26791 raft_consensus.cc:3037] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:23.920754 26935 raft_consensus.cc:3037] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:23.922961 26791 raft_consensus.cc:2454] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf7499ebc8ee46babe4ef86c6e1b9457 in term 1.
I20260927 07:03:23.922947 26935 raft_consensus.cc:2454] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf7499ebc8ee46babe4ef86c6e1b9457 in term 1.
I20260927 07:03:23.923358 27002 leader_election.cc:302] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf7499ebc8ee46babe4ef86c6e1b9457, e3753f0db1c54cabb581896a612a332a; no voters:
I20260927 07:03:23.923643 27151 raft_consensus.cc:2781] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:03:23.924059 27151 raft_consensus.cc:686] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 1 LEADER]: Becoming Leader. State: Replica: bf7499ebc8ee46babe4ef86c6e1b9457, State: Running, Role: LEADER
I20260927 07:03:23.924512 27151 consensus_queue.cc:237] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } }
I20260927 07:03:23.929219 26636 catalog_manager.cc:5495] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf7499ebc8ee46babe4ef86c6e1b9457 (127.13.121.195). New cstate: current_term: 1 leader_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf7499ebc8ee46babe4ef86c6e1b9457" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 38611 } health_report { overall_health: HEALTHY } } }
W20260927 07:03:23.953421 26985 tablet.cc:2369] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:03:23.981503 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:23.976032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58370 (local address 127.13.121.195:38611)
0927 07:03:23.976202 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:23.976206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:23.976225 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:03:23.976335 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:23.976338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:23.976420 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:23.976584 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:03:23.976903 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.977688 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.979246 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:23.979704 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:23.979816 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:23.980369 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:03:23.980964 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:03:23.981080 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:23.981209 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 07:03:23.981285 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 07:03:23.989913 26935 raft_consensus.cc:1261] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [term 1 FOLLOWER]: Refusing update from remote peer bf7499ebc8ee46babe4ef86c6e1b9457: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:23.989912 26791 raft_consensus.cc:1261] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [term 1 FOLLOWER]: Refusing update from remote peer bf7499ebc8ee46babe4ef86c6e1b9457: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:23.990969 27151 consensus_queue.cc:1035] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3753f0db1c54cabb581896a612a332a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 43295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:03:23.991278 27153 consensus_queue.cc:1035] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "037b7b469db043cb83a00c4db7e57626" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:03:24.013027 27157 mvcc.cc:205] Tried to move back new op lower bound from 7333857705932300288 to 7333857705678528512. Current Snapshot: MvccSnapshot[applied={T|T < 7333857705932300288}]
I20260927 07:03:24.026216 26637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:03:24.026482 26637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:03:24.030017 26637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8275dde0af624d7884d2578b636ba491: Sending DeleteTablet for 3 replicas of tablet c7655708d0264fdda9fe0cd8eef71aa5
I20260927 07:03:24.031165 26771 tablet_service.cc:1514] Processing DeleteTablet for tablet c7655708d0264fdda9fe0cd8eef71aa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:24 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:48914
I20260927 07:03:24.031276 27050 tablet_service.cc:1514] Processing DeleteTablet for tablet c7655708d0264fdda9fe0cd8eef71aa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:24 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:58362
I20260927 07:03:24.031383 26915 tablet_service.cc:1514] Processing DeleteTablet for tablet c7655708d0264fdda9fe0cd8eef71aa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:24 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:40772
I20260927 07:03:24.040829 27166 tablet_replica.cc:330] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: stopping tablet replica
I20260927 07:03:24.041111 27168 tablet_replica.cc:330] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: stopping tablet replica
I20260927 07:03:24.041158 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 26675
I20260927 07:03:24.041487 27166 raft_consensus.cc:2229] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:03:24.041639 27168 raft_consensus.cc:2229] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:03:24.042156 27166 raft_consensus.cc:2258] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:24.043088 27167 tablet_replica.cc:330] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: stopping tablet replica
I20260927 07:03:24.044068 27167 raft_consensus.cc:2229] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:03:24.044833 27167 raft_consensus.cc:2258] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:24.045351 27166 ts_tablet_manager.cc:1905] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:24.045538 27168 raft_consensus.cc:2258] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:24.047792 27167 ts_tablet_manager.cc:1905] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:24.055284 27167 ts_tablet_manager.cc:1918] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:24.055483 27167 log.cc:1196] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/c7655708d0264fdda9fe0cd8eef71aa5
I20260927 07:03:24.057144 27168 ts_tablet_manager.cc:1905] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:24.058467 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.042036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.13.121.193:43295)
0927 07:03:24.042241 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:24.042247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:24.043515 (+ 1268us) server_negotiation.cc:407] Connection header received
0927 07:03:24.043732 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:24.043737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:24.043818 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:24.043918 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:24.045941 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:24.046742 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:24.048568 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:24.049134 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:24.049245 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:24.052353 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:24.052365 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:24.052380 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:24.052419 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:24.055121 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:24.056365 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:24.056373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:24.056375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:24.056507 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:24.056881 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:24.056886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:24.056889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:24.057282 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:24.057575 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:24.057849 (+ 274us) server_negotiation.cc:299] Negotiation successful
0927 07:03:24.058103 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:03:24.059115 26831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:24.063091 27167 ts_tablet_manager.cc:1939] T c7655708d0264fdda9fe0cd8eef71aa5 P bf7499ebc8ee46babe4ef86c6e1b9457: Deleting consensus metadata
I20260927 07:03:24.064220 27168 ts_tablet_manager.cc:1918] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:24.064532 27168 log.cc:1196] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/c7655708d0264fdda9fe0cd8eef71aa5
I20260927 07:03:24.065264 26624 catalog_manager.cc:4841] TS bf7499ebc8ee46babe4ef86c6e1b9457 (127.13.121.195:38611): tablet c7655708d0264fdda9fe0cd8eef71aa5 (table test-table [id=f412da62340f4f4ab37b66c75430323b]) successfully deleted
I20260927 07:03:24.065771 27168 ts_tablet_manager.cc:1939] T c7655708d0264fdda9fe0cd8eef71aa5 P 037b7b469db043cb83a00c4db7e57626: Deleting consensus metadata
I20260927 07:03:24.067792 26624 catalog_manager.cc:4841] TS 037b7b469db043cb83a00c4db7e57626 (127.13.121.194:39797): tablet c7655708d0264fdda9fe0cd8eef71aa5 (table test-table [id=f412da62340f4f4ab37b66c75430323b]) successfully deleted
I20260927 07:03:24.159576 27166 ts_tablet_manager.cc:1918] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:24.159854 27166 log.cc:1196] T c7655708d0264fdda9fe0cd8eef71aa5 P e3753f0db1c54cabb581896a612a332a: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/c7655708d0264fdda9fe0cd8eef71aa5
I20260927 07:03:24.160156 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 26675
I20260927 07:03:24.174808 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 26840
W20260927 07:03:24.174925 26622 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.121.193:43295 (error 108)
I20260927 07:03:24.176812 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.176303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.0.0.1:48926)
0927 07:03:24.176493 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:24.176623 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260927 07:03:24.177196 26622 catalog_manager.cc:4584] TS e3753f0db1c54cabb581896a612a332a (127.13.121.193:43295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c7655708d0264fdda9fe0cd8eef71aa5: Network error: Client connection negotiation failed: client connection to 127.13.121.193:43295: connect: Connection refused (error 111)
I20260927 07:03:24.189338 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.176024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40788 (local address 127.13.121.194:39797)
0927 07:03:24.176199 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:24.176205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:24.176226 (+ 21us) server_negotiation.cc:407] Connection header received
0927 07:03:24.176293 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:24.176299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:24.176378 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:24.176476 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:24.177967 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:24.178673 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:24.180079 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:24.180481 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:24.180630 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:24.183291 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:24.183307 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:24.183323 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:24.183366 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:24.186343 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:24.187097 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:24.187105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:24.187107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:24.187340 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:24.187690 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:24.187695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:24.187698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:24.188071 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:24.188348 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:24.188679 (+ 331us) server_negotiation.cc:299] Negotiation successful
0927 07:03:24.188895 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:03:24.189924 26976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:24.230831 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.230285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.0.0.1:48942)
0927 07:03:24.230566 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:24.230661 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260927 07:03:24.232124 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.231673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.0.0.1:48944)
0927 07:03:24.231858 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:24.231952 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260927 07:03:24.280361 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 26840
I20260927 07:03:24.292889 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.292357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.0.0.1:48954)
0927 07:03:24.292590 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:24.292706 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260927 07:03:24.294137 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.293708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.0.0.1:48958)
0927 07:03:24.293872 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:24.293959 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260927 07:03:24.295074 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 26984
I20260927 07:03:24.312547 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.295872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.13.121.195:38611)
0927 07:03:24.296078 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:24.296084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:24.296105 (+ 21us) server_negotiation.cc:407] Connection header received
0927 07:03:24.296171 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:24.296176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:24.296275 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:24.296396 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:24.297710 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:24.298944 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:24.300551 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:24.301214 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:24.301358 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:24.303999 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:24.304017 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:24.304038 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:24.304094 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:24.309028 (+ 4934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:24.310051 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:24.310075 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:24.310079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:24.310251 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:24.310671 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:24.310676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:24.310679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:24.311203 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:24.311514 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:24.311911 (+ 397us) server_negotiation.cc:299] Negotiation successful
0927 07:03:24.312152 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 07:03:24.313470 27110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:24.374953 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.374443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.0.0.1:48970)
0927 07:03:24.374676 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:24.374801 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260927 07:03:24.376057 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.375640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.0.0.1:48984)
0927 07:03:24.375814 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:24.375893 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260927 07:03:24.433576 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 26984
I20260927 07:03:24.448799 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 26605
I20260927 07:03:24.450104 26668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:24.561399 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:24.560771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:43295 (local address 127.0.0.1:49000)
0927 07:03:24.561119 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:24.561226 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
I20260927 07:03:24.561491 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 26605
I20260927 07:03:24.579417 13799 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 07:03:24.915288 13799 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 07:03:24.915489 13799 mini_ranger.cc:66] Stopping Ranger...
I20260927 07:03:25.251278 13799 mini_ranger.cc:68] Stopped Ranger
2026-09-27 07:03:25.251 UTC [25657] LOG: received smart shutdown request
2026-09-27 07:03:25.254 UTC [25657] LOG: background worker "logical replication launcher" (PID 25666) exited with exit code 1
2026-09-27 07:03:25.255 UTC [25661] LOG: shutting down
2026-09-27 07:03:25.431 UTC [25657] LOG: database system is shut down
2026-09-27T07:03:25Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54214 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:03:25 dist-test-slave-p6hz krb5kdc[27182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:03:25 dist-test-slave-p6hz krb5kdc[27182](info): set up 2 sockets
Sep 27 07:03:25 dist-test-slave-p6hz krb5kdc[27182](info): commencing operation
krb5kdc: starting...
W20260927 07:03:26.551446 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:03:26 dist-test-slave-p6hz krb5kdc[27182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:03:26.776185 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:42257
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:42257,127.13.121.253:41665,127.13.121.252:46401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:27.103492 27195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:27.103834 27195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:27.103967 27195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:27.111333 27195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:27.111464 27195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:27.111618 27195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:27.111698 27195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:27.111759 27195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:27.122006 27195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:42257,127.13.121.253:41665,127.13.121.252:46401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:42257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:27.123530 27195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:27.124877 27195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:27.136825 27200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:27.136857 27201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:27.136878 27203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:27.137948 27195 server_base.cc:1034] running on GCE node
I20260927 07:03:27.138765 27195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:27.140337 27195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:27.141585 27195 hybrid_clock.cc:648] HybridClock initialized: now 1790492607141531 us; error 38 us; skew 500 ppm
I20260927 07:03:27.141959 27195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:27 dist-test-slave-p6hz krb5kdc[27182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492607, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:27.154433 27195 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:03:27.156134 27195 webserver.cc:460] Webserver started at http://127.13.121.254:43837/ using document root <none> and password file <none>
I20260927 07:03:27.156906 27195 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:27.157037 27195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:27.157354 27195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:27.160629 27195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "800904235b724421b218e3eb1fcf7554"
format_stamp: "Formatted at 2026-09-27 07:03:27 on dist-test-slave-p6hz"
server_key: "09474946ef63d6e3ac80baf412a82675"
server_key_iv: "268b10e3290668f3a42db007fb91d8fd"
server_key_version: "encryptionkey@0"
I20260927 07:03:27.161482 27195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "800904235b724421b218e3eb1fcf7554"
format_stamp: "Formatted at 2026-09-27 07:03:27 on dist-test-slave-p6hz"
server_key: "09474946ef63d6e3ac80baf412a82675"
server_key_iv: "268b10e3290668f3a42db007fb91d8fd"
server_key_version: "encryptionkey@0"
I20260927 07:03:27.166985 27195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:03:27.170650 27210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:27.172293 27195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:03:27.172521 27195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "800904235b724421b218e3eb1fcf7554"
format_stamp: "Formatted at 2026-09-27 07:03:27 on dist-test-slave-p6hz"
server_key: "09474946ef63d6e3ac80baf412a82675"
server_key_iv: "268b10e3290668f3a42db007fb91d8fd"
server_key_version: "encryptionkey@0"
I20260927 07:03:27.172734 27195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:27.201944 27195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:27.206266 27195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:27.206547 27195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:27.224395 27195 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:42257
I20260927 07:03:27.224413 27261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:42257 every 8 connection(s)
I20260927 07:03:27.225747 27195 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:03:27.227962 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 27195
I20260927 07:03:27.228183 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:03:27.228905 13799 external_mini_cluster.cc:1428] Setting key 236d636cc549fcc986aa90de38820c5f
I20260927 07:03:27.230039 27262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:27.238287 27262 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.121.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.253@KRBTEST.COM" created.
Sep 27 07:03:27 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492607, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:27.259939 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.241307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:42257 (local address 127.0.0.1:59280)
0927 07:03:27.242126 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.242171 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:27.244192 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:27.244998 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:27.245005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:27.245671 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:27.246001 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:27.246018 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.247507 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.247511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:27.248608 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:27.248621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.249516 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.249523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:27.249705 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:27.250296 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:27.250324 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:27.252735 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:27.255919 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:27.255932 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:27.255945 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:27.256470 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:27.256860 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:27.256865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:27.256869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:27.257064 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:27.257980 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:27.257987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:27.258188 (+ 201us) client_negotiation.cc:769] Sending connection context
0927 07:03:27.258398 (+ 210us) client_negotiation.cc:240] Negotiation successful
0927 07:03:27.258827 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":350,"threads_started":1}
I20260927 07:03:27.259994 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.241308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59280 (local address 127.13.121.254:42257)
0927 07:03:27.242925 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:27.242933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:27.242980 (+ 47us) server_negotiation.cc:407] Connection header received
0927 07:03:27.244552 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:27.244558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:27.244789 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:27.244959 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:27.246186 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.247374 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.248729 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.249301 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.249484 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:27.252963 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:27.252971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:27.252984 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:27.253013 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:27.255780 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:27.256596 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:27.256603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:27.256605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:27.256751 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:27.257167 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:27.257170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:27.257172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:27.257814 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:27.258021 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:27.258568 (+ 547us) server_negotiation.cc:299] Negotiation successful
0927 07:03:27.259023 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":1415,"spinlock_wait_cycles":15744,"thread_start_us":1209,"threads_started":1}
I20260927 07:03:27.262797 27262 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:03:27.263942 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.263476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:39612)
0927 07:03:27.263656 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.263795 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260927 07:03:27.264214 27214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
I20260927 07:03:27.265945 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.265493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:39614)
0927 07:03:27.265696 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.265786 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260927 07:03:27.266170 27262 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:41665: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab.
Entry for principal kudu/127.13.121.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab.
I20260927 07:03:27.278940 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.253:41665
--webserver_interface=127.13.121.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab
--principal=kudu/127.13.121.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:42257,127.13.121.253:41665,127.13.121.252:46401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:03:27.315513 27262 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } attempt: 1
I20260927 07:03:27.316731 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.316171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:39630)
0927 07:03:27.316414 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.316550 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260927 07:03:27.317801 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.317377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:39638)
0927 07:03:27.317533 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.317625 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 07:03:27.318009 27262 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:41665: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
I20260927 07:03:27.386263 27262 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } attempt: 2
I20260927 07:03:27.387547 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.386946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:39648)
0927 07:03:27.387255 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.387371 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260927 07:03:27.388643 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.388220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:39664)
0927 07:03:27.388370 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.388481 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260927 07:03:27.388855 27262 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:41665: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
I20260927 07:03:27.480118 27262 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } attempt: 3
I20260927 07:03:27.481253 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.480744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:39676)
0927 07:03:27.480970 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.481087 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 07:03:27.482415 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.481953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:39678)
0927 07:03:27.482118 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.482233 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260927 07:03:27.482646 27262 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:41665: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
W20260927 07:03:27.599608 27269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:27.599889 27269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:27.599977 27269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:27.607107 27269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:27.607250 27269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:27.607300 27269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:27.607355 27269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:27.607402 27269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:27.617395 27269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:42257,127.13.121.253:41665,127.13.121.252:46401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab
--principal=kudu/127.13.121.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.253:41665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.121.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:27.618778 27269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:27.620116 27269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 07:03:27.625880 27262 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } attempt: 4
I20260927 07:03:27.627339 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.626631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:47710)
0927 07:03:27.627064 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.627189 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275}
I20260927 07:03:27.628453 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.628078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:47718)
0927 07:03:27.628233 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.628331 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260927 07:03:27.628665 27262 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:41665: Network error: Client connection negotiation failed: client connection to 127.13.121.253:41665: connect: Connection refused (error 111)
W20260927 07:03:27.632781 27274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:27.632846 27275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:27.632814 27277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:27.633423 27269 server_base.cc:1034] running on GCE node
I20260927 07:03:27.634238 27269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:27.635747 27269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:27.636961 27269 hybrid_clock.cc:648] HybridClock initialized: now 1790492607636910 us; error 43 us; skew 500 ppm
I20260927 07:03:27.637308 27269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:27 dist-test-slave-p6hz krb5kdc[27182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492607, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:27.640542 27269 init.cc:377] Logged in from keytab as kudu/127.13.121.253@KRBTEST.COM (short username kudu)
I20260927 07:03:27.642004 27269 webserver.cc:460] Webserver started at http://127.13.121.253:44581/ using document root <none> and password file <none>
I20260927 07:03:27.642730 27269 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:27.642858 27269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:27.643179 27269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:27.646229 27269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/data/instance:
uuid: "7764c61f07ca489797fff7ed5d96889f"
format_stamp: "Formatted at 2026-09-27 07:03:27 on dist-test-slave-p6hz"
server_key: "a6b266c6278917a92d921bd9b695057d"
server_key_iv: "37f0631059a6f67e003a0f14bb5dc81c"
server_key_version: "encryptionkey@0"
I20260927 07:03:27.647035 27269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/wal/instance:
uuid: "7764c61f07ca489797fff7ed5d96889f"
format_stamp: "Formatted at 2026-09-27 07:03:27 on dist-test-slave-p6hz"
server_key: "a6b266c6278917a92d921bd9b695057d"
server_key_iv: "37f0631059a6f67e003a0f14bb5dc81c"
server_key_version: "encryptionkey@0"
I20260927 07:03:27.652617 27269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:03:27.656315 27284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:27.657785 27269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:03:27.657961 27269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/wal
uuid: "7764c61f07ca489797fff7ed5d96889f"
format_stamp: "Formatted at 2026-09-27 07:03:27 on dist-test-slave-p6hz"
server_key: "a6b266c6278917a92d921bd9b695057d"
server_key_iv: "37f0631059a6f67e003a0f14bb5dc81c"
server_key_version: "encryptionkey@0"
I20260927 07:03:27.658164 27269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:27.668969 27269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:27.673278 27269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:27.673563 27269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:27.690264 27269 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.253:41665
I20260927 07:03:27.690291 27335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.253:41665 every 8 connection(s)
I20260927 07:03:27.691697 27269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
I20260927 07:03:27.695425 27336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:27.699335 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 27269
I20260927 07:03:27.699502 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-1/wal/instance
I20260927 07:03:27.699898 13799 external_mini_cluster.cc:1428] Setting key 8c984cec0da33d8307b831f39cbf2f57
I20260927 07:03:27.701931 27336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:03:27 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492607, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.252@KRBTEST.COM" created.
I20260927 07:03:27.724216 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.704670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.13.121.254:42257)
0927 07:03:27.704842 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:27.704846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:27.706207 (+ 1361us) server_negotiation.cc:407] Connection header received
0927 07:03:27.708760 (+ 2553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:27.708765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:27.708835 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:27.708920 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:27.710881 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.711671 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.713737 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.714187 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.714296 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:27.718391 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:27.718396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:27.718400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:27.718436 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:27.721096 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:27.721856 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:27.721866 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:27.721868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:27.721988 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:27.722401 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:27.722404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:27.722406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:27.722709 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:27.722852 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:27.723663 (+ 811us) server_negotiation.cc:299] Negotiation successful
0927 07:03:27.723900 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 07:03:27.725301 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.705174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:42257 (local address 127.0.0.1:34842)
0927 07:03:27.706015 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.706064 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:27.708453 (+ 2389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:27.709202 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:27.709214 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:27.709928 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:27.710674 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:27.710697 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.711843 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.711849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:27.713573 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:27.713588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.714409 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.714416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:27.714717 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:27.715467 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:27.715500 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:27.718095 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:27.721250 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:27.721260 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:27.721280 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:27.721732 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:27.722095 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:27.722100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:27.722104 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:27.722291 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:27.722837 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:27.722845 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:27.723278 (+ 433us) client_negotiation.cc:769] Sending connection context
0927 07:03:27.723548 (+ 270us) client_negotiation.cc:240] Negotiation successful
0927 07:03:27.724096 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":352,"threads_started":1}
I20260927 07:03:27.727156 27336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:03:27 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492607, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:03:27.741360 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.727953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:47722)
0927 07:03:27.728160 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.728178 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:27.728290 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:27.729194 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:27.729198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:27.729615 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:27.729745 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:27.729753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.731031 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.731037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:27.732127 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:27.732139 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.732913 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.732918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:27.733095 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:27.733656 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:27.733672 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:27.735862 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:27.738916 (+ 3054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:27.738921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:27.738926 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:27.739329 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:27.739669 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:27.739674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:27.739678 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:27.739811 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:27.740425 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:27.740429 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:27.740560 (+ 131us) client_negotiation.cc:769] Sending connection context
0927 07:03:27.740678 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 07:03:27.740941 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260927 07:03:27.741469 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.728168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47722 (local address 127.13.121.253:41665)
0927 07:03:27.728804 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:27.728809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:27.728833 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:03:27.728875 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:27.728881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:27.729054 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:27.729191 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:27.729890 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.730903 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.732246 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.732782 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.732889 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:27.736057 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:27.736069 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:27.736085 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:27.736120 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:27.738794 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:27.739448 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:27.739457 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:27.739461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:27.739569 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:27.739929 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:27.739932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:27.739934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:27.740310 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:27.740528 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:27.740850 (+ 322us) server_negotiation.cc:299] Negotiation successful
0927 07:03:27.741061 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":363,"threads_started":1}
I20260927 07:03:27.743911 27336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:03:27.744982 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.744520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40118)
0927 07:03:27.744689 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.744792 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260927 07:03:27.745211 27285 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
I20260927 07:03:27.746776 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.746392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40126)
0927 07:03:27.746584 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.746665 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 07:03:27.747051 27336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:46401: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab.
Entry for principal kudu/127.13.121.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab.
I20260927 07:03:27.750662 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.252:46401
--webserver_interface=127.13.121.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab
--principal=kudu/127.13.121.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:42257,127.13.121.253:41665,127.13.121.252:46401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:03:27.796389 27336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } attempt: 1
I20260927 07:03:27.797673 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.797195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40136)
0927 07:03:27.797394 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.797496 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 07:03:27.798584 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.798246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40140)
0927 07:03:27.798399 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.798466 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260927 07:03:27.798781 27336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:46401: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
I20260927 07:03:27.866994 27336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } attempt: 2
I20260927 07:03:27.868079 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.867631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40146)
0927 07:03:27.867822 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.867914 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260927 07:03:27.869035 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.868710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40150)
0927 07:03:27.868851 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.868923 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 07:03:27.869234 27336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:46401: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
I20260927 07:03:27.927925 27262 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } attempt: 5
Sep 27 07:03:27 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492607, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:03:27.942443 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.928654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:47728)
0927 07:03:27.928880 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.928913 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:27.929024 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:27.929404 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:27.929408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:27.929773 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:27.929943 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:27.929952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.931473 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.931477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:27.932399 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:27.932413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.933379 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.933384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:27.933537 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:27.934113 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:27.934132 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:27.936174 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:27.940040 (+ 3866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:27.940045 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:27.940047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:27.940425 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:27.940845 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:27.940849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:27.940852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:27.941006 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:27.941675 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:27.941678 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:27.941826 (+ 148us) client_negotiation.cc:769] Sending connection context
0927 07:03:27.941951 (+ 125us) client_negotiation.cc:240] Negotiation successful
0927 07:03:27.942190 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260927 07:03:27.942629 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.928786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.13.121.253:41665)
0927 07:03:27.928992 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:27.928998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:27.929023 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:03:27.929130 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:27.929135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:27.929233 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:27.929339 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:27.930118 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.931339 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.932538 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:27.933226 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:27.933353 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:27.936394 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:27.936401 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:27.936406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:27.936450 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:27.939852 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:27.940547 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:27.940553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:27.940557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:27.940718 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:27.941115 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:27.941120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:27.941124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:27.941562 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:27.941724 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:27.942083 (+ 359us) server_negotiation.cc:299] Negotiation successful
0927 07:03:27.942326 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:03:27.943653 27262 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:03:27.944665 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.944286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40152)
0927 07:03:27.944448 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.944525 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260927 07:03:27.945531 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.945218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40154)
0927 07:03:27.945362 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.945432 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260927 07:03:27.945714 27262 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:46401: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
I20260927 07:03:27.960438 27336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } attempt: 3
I20260927 07:03:27.961531 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.961047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40168)
0927 07:03:27.961227 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.961321 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260927 07:03:27.962597 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.962132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40172)
0927 07:03:27.962354 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.962440 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260927 07:03:27.962771 27336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:46401: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
I20260927 07:03:27.996968 27262 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } attempt: 1
I20260927 07:03:27.998157 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.997676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40174)
0927 07:03:27.997896 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.997997 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260927 07:03:27.999111 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:27.998774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40190)
0927 07:03:27.998920 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:27.998998 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 07:03:27.999336 27262 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:46401: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
I20260927 07:03:28.067539 27262 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } attempt: 2
I20260927 07:03:28.068758 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.068221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40202)
0927 07:03:28.068456 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:28.068547 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 07:03:28.069716 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.069367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40212)
0927 07:03:28.069543 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:28.069620 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 07:03:28.069909 27262 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:46401: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
W20260927 07:03:28.071988 27343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:28.072296 27343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:28.072422 27343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:28.079936 27343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:28.080060 27343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:28.080135 27343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:28.080210 27343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:28.080266 27343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:28.090163 27343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:42257,127.13.121.253:41665,127.13.121.252:46401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab
--principal=kudu/127.13.121.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.252:46401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.121.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:28.091495 27343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:28.092777 27343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 07:03:28.105970 27336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } attempt: 4
W20260927 07:03:28.107081 27348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:28.107049 27349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:28.107179 27351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:28.107424 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.106707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40214)
0927 07:03:28.107099 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:28.107231 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214}
I20260927 07:03:28.108533 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.108135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40226)
0927 07:03:28.108303 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:28.108388 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260927 07:03:28.108776 27336 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:46401: Network error: Client connection negotiation failed: client connection to 127.13.121.252:46401: connect: Connection refused (error 111)
I20260927 07:03:28.109524 27343 server_base.cc:1034] running on GCE node
I20260927 07:03:28.110337 27343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:28.111848 27343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:28.113072 27343 hybrid_clock.cc:648] HybridClock initialized: now 1790492608113019 us; error 48 us; skew 500 ppm
I20260927 07:03:28.113408 27343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:28 dist-test-slave-p6hz krb5kdc[27182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492608, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:28.116586 27343 init.cc:377] Logged in from keytab as kudu/127.13.121.252@KRBTEST.COM (short username kudu)
I20260927 07:03:28.118036 27343 webserver.cc:460] Webserver started at http://127.13.121.252:42975/ using document root <none> and password file <none>
I20260927 07:03:28.118734 27343 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:28.118834 27343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:28.119092 27343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:28.122164 27343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/data/instance:
uuid: "1e0d58df9dbe447bad12aab145064a45"
format_stamp: "Formatted at 2026-09-27 07:03:28 on dist-test-slave-p6hz"
server_key: "d016c1721bcab0d128ae3e07c13be502"
server_key_iv: "a43c0b627eb973122cb58faf7e979767"
server_key_version: "encryptionkey@0"
I20260927 07:03:28.122941 27343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/wal/instance:
uuid: "1e0d58df9dbe447bad12aab145064a45"
format_stamp: "Formatted at 2026-09-27 07:03:28 on dist-test-slave-p6hz"
server_key: "d016c1721bcab0d128ae3e07c13be502"
server_key_iv: "a43c0b627eb973122cb58faf7e979767"
server_key_version: "encryptionkey@0"
I20260927 07:03:28.128471 27343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260927 07:03:28.132153 27358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:28.133718 27343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260927 07:03:28.133898 27343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/wal
uuid: "1e0d58df9dbe447bad12aab145064a45"
format_stamp: "Formatted at 2026-09-27 07:03:28 on dist-test-slave-p6hz"
server_key: "d016c1721bcab0d128ae3e07c13be502"
server_key_iv: "a43c0b627eb973122cb58faf7e979767"
server_key_version: "encryptionkey@0"
I20260927 07:03:28.134102 27343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:28.148976 27343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:28.153342 27343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:28.153641 27343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:28.171172 27343 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.252:46401
I20260927 07:03:28.171206 27409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.252:46401 every 8 connection(s)
I20260927 07:03:28.172582 27343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
I20260927 07:03:28.176100 27262 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } attempt: 3
I20260927 07:03:28.176460 27410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:28.180866 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 27343
I20260927 07:03:28.181049 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790492406663968-13799-0/minicluster-data/master-2/wal/instance
I20260927 07:03:28.181564 13799 external_mini_cluster.cc:1428] Setting key fa3ceb5831e09afb0284142deb11cf28
I20260927 07:03:28.187201 27410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:03:28 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492607, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
Sep 27 07:03:28 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
I20260927 07:03:28.198416 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.176674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40230)
0927 07:03:28.176849 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:28.176893 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:28.177026 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:28.182241 (+ 5215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:28.182248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:28.182767 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:28.182990 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.183005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.185548 (+ 2543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.185552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.186459 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.186473 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.187638 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.187645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.187839 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.188493 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:28.188517 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:28.191201 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:28.195598 (+ 4397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.195604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.195608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.196036 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.196487 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.196491 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.196495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.196665 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.197485 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:28.197490 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:28.197671 (+ 181us) client_negotiation.cc:769] Sending connection context
0927 07:03:28.197788 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 07:03:28.198058 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 07:03:28.199724 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.177559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40230 (local address 127.13.121.252:46401)
0927 07:03:28.178290 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.178301 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.178339 (+ 38us) server_negotiation.cc:407] Connection header received
0927 07:03:28.181078 (+ 2739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.181115 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.181729 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.183297 (+ 1568us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.183914 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.185371 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.186603 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.187448 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.187654 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.191510 (+ 3856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.191521 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.191538 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.191587 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.195444 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.196162 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.196170 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.196179 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.196368 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.196757 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.196763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.196766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.197354 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.197535 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.198077 (+ 542us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.198506 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"spinlock_wait_cycles":14592,"thread_start_us":337,"threads_started":1}
Sep 27 07:03:28 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492608, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:28.203838 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.183040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.13.121.252:46401)
0927 07:03:28.187656 (+ 4616us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.187660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.187685 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:03:28.187730 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.187735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.187808 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.187924 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.189423 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.190505 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.192380 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.193090 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.193230 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.197285 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.197291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.197294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.197378 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.200404 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.201046 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.201051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.201055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.201229 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.201643 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.201649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.201653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.201930 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.202217 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.203268 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.203535 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4460,"spinlock_wait_cycles":14080,"thread_start_us":4275,"threads_started":1}
I20260927 07:03:28.207322 27262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554: Bootstrap starting.
I20260927 07:03:28.207887 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.190324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.13.121.254:42257)
0927 07:03:28.190488 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.190494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.191254 (+ 760us) server_negotiation.cc:407] Connection header received
0927 07:03:28.191533 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.191537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.191593 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.191670 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.192887 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.193970 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.195903 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.196444 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.196549 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.201664 (+ 5115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.201671 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.201675 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.201711 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.205076 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.205737 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.205742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.205745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.205914 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.206298 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.206302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.206306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.206664 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.206814 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.207264 (+ 450us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.207480 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 07:03:28.207909 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.190296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:42257 (local address 127.0.0.1:34846)
0927 07:03:28.191107 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:28.191178 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:28.191362 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:28.191728 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:28.191735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:28.192392 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:28.192741 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.192750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.194156 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.194161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.195758 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.195771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.196614 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.196621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.196793 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.197973 (+ 1180us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:28.198003 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:28.201454 (+ 3451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:28.205207 (+ 3753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.205218 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.205223 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.205612 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.206047 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.206051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.206053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.206184 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.206770 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:28.206776 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:28.206929 (+ 153us) client_negotiation.cc:769] Sending connection context
0927 07:03:28.207084 (+ 155us) client_negotiation.cc:240] Negotiation successful
0927 07:03:28.207399 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":475,"threads_started":1}
I20260927 07:03:28.209470 27410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:03:28.212111 27262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:28.213479 27262 log.cc:824] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:28.216938 27262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554: No bootstrap required, opened a new log
Sep 27 07:03:28 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492608, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
Sep 27 07:03:28 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:28.225518 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.210302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47742 (local address 127.13.121.253:41665)
0927 07:03:28.210448 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.210452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.211372 (+ 920us) server_negotiation.cc:407] Connection header received
0927 07:03:28.211516 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.211521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.211589 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.211669 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.212372 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.213035 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.214334 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.214969 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.215085 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.218767 (+ 3682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.218774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.218779 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.218822 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.222198 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.223069 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.223076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.223079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.223267 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.223749 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.223754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.223758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.224124 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.224252 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.224829 (+ 577us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.225081 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:03:28.226418 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.209420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34860 (local address 127.13.121.254:42257)
0927 07:03:28.209592 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.209597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.209616 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:03:28.209713 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.209717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.209785 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.209879 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.211433 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.212568 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.214320 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.214964 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.215067 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.219453 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.219460 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.219465 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.219506 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.222725 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.223531 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.223536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.223539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.223727 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.224200 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.224205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.224208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.224581 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.224881 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.226001 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.226183 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 07:03:28.226563 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.210309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:41665 (local address 127.0.0.1:47742)
0927 07:03:28.211284 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:28.211299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:28.211396 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:28.211703 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:28.211708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:28.212056 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:28.212194 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.212204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.213199 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.213205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.214199 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.214214 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.215170 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.215176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.215391 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.215974 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:28.215996 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:28.218515 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:28.222360 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.222367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.222371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.222931 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.223431 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.223436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.223440 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.223612 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.224349 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:28.224353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:28.224531 (+ 178us) client_negotiation.cc:769] Sending connection context
0927 07:03:28.225900 (+ 1369us) client_negotiation.cc:240] Negotiation successful
0927 07:03:28.226217 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878}
I20260927 07:03:28.227891 27410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:03:28.227772 27262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } }
I20260927 07:03:28.228183 27262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:28.228298 27262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800904235b724421b218e3eb1fcf7554, State: Initialized, Role: FOLLOWER
I20260927 07:03:28.230012 27262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } }
Sep 27 07:03:28 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492608, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
Sep 27 07:03:28 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:03:28.240785 27417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.241290 27417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:03:28.242347 27262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:28.243685 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.228708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.13.121.252:46401)
0927 07:03:28.228866 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.228872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.229036 (+ 164us) server_negotiation.cc:407] Connection header received
0927 07:03:28.229176 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.229181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.229246 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.229330 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.230281 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.231323 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.232908 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.233509 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.233588 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.237309 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.237315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.237320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.237355 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.240426 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.241148 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.241154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.241158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.241293 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.241658 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.241663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.241667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.242054 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.242225 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.242924 (+ 699us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.243357 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":201,"server-negotiator.queue_time_us":28}
I20260927 07:03:28.243826 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.228575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40252)
0927 07:03:28.228966 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:28.228982 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:28.229078 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:28.229481 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:28.229484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:28.229950 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:28.230141 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.230153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.231460 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.231465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.232786 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.232798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.233817 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.233822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.233987 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.234596 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:28.234625 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:28.237134 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:28.240563 (+ 3429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.240568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.240571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.241030 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.241423 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.241427 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.241430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.241557 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.242207 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:28.242213 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:28.242634 (+ 421us) client_negotiation.cc:769] Sending connection context
0927 07:03:28.242820 (+ 186us) client_negotiation.cc:240] Negotiation successful
0927 07:03:28.243189 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309}
I20260927 07:03:28.247655 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.228485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47758 (local address 127.13.121.253:41665)
0927 07:03:28.228658 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.228664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.228892 (+ 228us) server_negotiation.cc:407] Connection header received
0927 07:03:28.229038 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.229044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.229114 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.229195 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.230897 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.232099 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.234125 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.235184 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.235285 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.241773 (+ 6488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.241778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.241783 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.241824 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.245056 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.245715 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.245721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.245725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.245869 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.246228 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.246234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.246237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.246501 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.246767 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.247064 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.247341 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 07:03:28.250465 27410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45: Bootstrap starting.
I20260927 07:03:28.252357 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:03:28.254762 27410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:28.256145 27410 log.cc:824] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45: Log is configured to *not* fsync() on all Append() calls
W20260927 07:03:28.263788 27428 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:03:28.264027 27428 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:03:28.264614 27410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45: No bootstrap required, opened a new log
I20260927 07:03:28.258138 27195 master_runner.cc:186] Error getting master registration for 127.13.121.253:41665: OK, instance_id { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" instance_seqno: 1790492607685278 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:03:28.270493 27195 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:03:28.274302 27410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } }
I20260927 07:03:28.274798 27410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:28.274940 27410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e0d58df9dbe447bad12aab145064a45, State: Initialized, Role: FOLLOWER
I20260927 07:03:28.275640 27410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } }
I20260927 07:03:28.288221 27417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:28.288529 27417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } }
I20260927 07:03:28.290258 27417 leader_election.cc:288] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7764c61f07ca489797fff7ed5d96889f (127.13.121.253:41665), 1e0d58df9dbe447bad12aab145064a45 (127.13.121.252:46401)
I20260927 07:03:28.299247 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.273028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.13.121.254:42257)
0927 07:03:28.273210 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.273216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.273235 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:03:28.273463 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.273469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.273551 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.273639 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.276189 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.277404 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.279483 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.280193 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.280305 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.289722 (+ 9417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.289728 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.289734 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.289805 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.293438 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.294317 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.294324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.294327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.294510 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.296739 (+ 2229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.296746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.296749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.297150 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.297396 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.298620 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.298865 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:03:28.305263 27410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:28.305311 27385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "800904235b724421b218e3eb1fcf7554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e0d58df9dbe447bad12aab145064a45" is_pre_election: true
I20260927 07:03:28.305976 27385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 800904235b724421b218e3eb1fcf7554 in term 0.
I20260927 07:03:28.306748 27211 leader_election.cc:302] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e0d58df9dbe447bad12aab145064a45, 800904235b724421b218e3eb1fcf7554; no voters:
I20260927 07:03:28.307222 27417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:03:28.307370 27417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:03:28.307469 27417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:28.310716 27417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } }
I20260927 07:03:28.311062 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.274029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40260 (local address 127.13.121.252:46401)
0927 07:03:28.274197 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.274202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.281816 (+ 7614us) server_negotiation.cc:407] Connection header received
0927 07:03:28.283446 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.283454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.283568 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.283649 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.286874 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.287918 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.294969 (+ 7051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.295675 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.295789 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.304320 (+ 8531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.304329 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.304333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.304382 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.307907 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.308718 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.308724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.308728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.308887 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.309323 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.309329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.309332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.309716 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.309924 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.310474 (+ 550us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.310705 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 07:03:28.312754 27385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "800904235b724421b218e3eb1fcf7554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e0d58df9dbe447bad12aab145064a45"
I20260927 07:03:28.312963 27385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:28.316298 27385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 800904235b724421b218e3eb1fcf7554 in term 1.
I20260927 07:03:28.316582 27417 leader_election.cc:288] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [CANDIDATE]: Term 1 election: Requested vote from peers 7764c61f07ca489797fff7ed5d96889f (127.13.121.253:41665), 1e0d58df9dbe447bad12aab145064a45 (127.13.121.252:46401)
I20260927 07:03:28.316820 27211 leader_election.cc:302] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e0d58df9dbe447bad12aab145064a45, 800904235b724421b218e3eb1fcf7554; no voters:
I20260927 07:03:28.317121 27417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:03:28.317729 27417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [term 1 LEADER]: Becoming Leader. State: Replica: 800904235b724421b218e3eb1fcf7554, State: Running, Role: LEADER
I20260927 07:03:28.318398 27417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } }
I20260927 07:03:28.305024 27437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.321811 27437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:03:28.322953 27440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 800904235b724421b218e3eb1fcf7554. Latest consensus state: current_term: 1 leader_uuid: "800904235b724421b218e3eb1fcf7554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.323374 27440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:28.329988 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.279486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.13.121.253:41665)
0927 07:03:28.279658 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.279664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.284137 (+ 4473us) server_negotiation.cc:407] Connection header received
0927 07:03:28.284588 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.284594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.284677 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.284747 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.286628 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.287764 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.296181 (+ 8417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.296790 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.296887 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.320749 (+ 23862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.320757 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.320762 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.320815 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.324687 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.325997 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.326003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.326007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.326207 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.326996 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.327001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.327004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.327530 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.327777 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.328925 (+ 1148us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.329291 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 07:03:28.335065 27443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:03:28.337577 27443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:03:28.353123 27385 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [term 1 FOLLOWER]: Refusing update from remote peer 800904235b724421b218e3eb1fcf7554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:28.354108 27417 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:03:28.368610 27437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 800904235b724421b218e3eb1fcf7554. Latest consensus state: current_term: 1 leader_uuid: "800904235b724421b218e3eb1fcf7554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.368963 27437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:03:28.375073 27445 mvcc.cc:205] Tried to move back new op lower bound from 7333857723803099136 to 7333857723676942336. Current Snapshot: MvccSnapshot[applied={T|T < 7333857723803099136}]
I20260927 07:03:28.375787 27448 mvcc.cc:205] Tried to move back new op lower bound from 7333857723803099136 to 7333857723676942336. Current Snapshot: MvccSnapshot[applied={T|T < 7333857723803099136}]
I20260927 07:03:28.389405 27417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "800904235b724421b218e3eb1fcf7554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.389779 27417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:28.390235 27437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "800904235b724421b218e3eb1fcf7554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.390515 27437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:03:28.392645 27443 catalog_manager.cc:1347] Generated new cluster ID: 04c766bafcb4436482c2b5d92a8dc624
I20260927 07:03:28.392753 27443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:03:28.408018 27336 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } attempt: 5
Sep 27 07:03:28 dist-test-slave-p6hz krb5kdc[27182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492607, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
I20260927 07:03:28.418767 27457 catalog_manager.cc:1259] Loaded cluster ID: 04c766bafcb4436482c2b5d92a8dc624
I20260927 07:03:28.418864 27457 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45: loading cluster ID for follower catalog manager: success
W20260927 07:03:28.420185 27457 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260927 07:03:28.420327 27457 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:03:28.423171 27443 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:03:28.426186 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.408597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:46401 (local address 127.0.0.1:40270)
0927 07:03:28.408828 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:28.408847 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:28.408955 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:28.409409 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:28.409414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:28.409989 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:28.410216 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.410231 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.411735 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.411741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.412978 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:28.412993 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.414045 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.414050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:28.414223 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.414819 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:28.414842 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:28.417547 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:28.422966 (+ 5419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.422972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.422976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.423626 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.424124 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:28.424129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:28.424134 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:28.424346 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:28.425068 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:28.425073 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:28.425263 (+ 190us) client_negotiation.cc:769] Sending connection context
0927 07:03:28.425394 (+ 131us) client_negotiation.cc:240] Negotiation successful
0927 07:03:28.425741 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260927 07:03:28.428208 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:28.408668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.13.121.252:46401)
0927 07:03:28.408860 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:28.408866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:28.408886 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:03:28.409091 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:28.409097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:28.409177 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:28.409250 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:28.410385 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.411544 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.413163 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:28.413854 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:28.413972 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:28.417750 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:28.417757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:28.417761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:28.417803 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:28.421876 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.423791 (+ 1915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.423797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.423801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.423983 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:28.424503 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:28.424509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:28.424513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:28.424907 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:28.426923 (+ 2016us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:28.427295 (+ 372us) server_negotiation.cc:299] Negotiation successful
0927 07:03:28.427562 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 07:03:28.434736 27336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f: Bootstrap starting.
I20260927 07:03:28.439698 27336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:28.441431 27336 log.cc:824] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:28.441921 27443 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:03:28.464023 27336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f: No bootstrap required, opened a new log
I20260927 07:03:28.410955 27343 master_runner.cc:186] Error getting master registration for 127.13.121.253:41665: OK, instance_id { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" instance_seqno: 1790492607685278 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:03:28.472452 27343 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:03:28.476934 27336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } }
I20260927 07:03:28.477525 27336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:28.477651 27336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7764c61f07ca489797fff7ed5d96889f, State: Initialized, Role: FOLLOWER
I20260927 07:03:28.478443 27336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } }
I20260927 07:03:28.485431 27460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.486709 27336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:28.487212 27309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:28.487290 27443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554: Generated new TSK 0
I20260927 07:03:28.488404 27443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:03:28.491034 27309 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [term 1 FOLLOWER]: Refusing update from remote peer 800904235b724421b218e3eb1fcf7554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:28.491245 27311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "800904235b724421b218e3eb1fcf7554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7764c61f07ca489797fff7ed5d96889f" is_pre_election: true
I20260927 07:03:28.492053 27310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "800904235b724421b218e3eb1fcf7554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7764c61f07ca489797fff7ed5d96889f"
I20260927 07:03:28.496978 27417 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:03:28.503746 27460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:03:28.551021 27460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 800904235b724421b218e3eb1fcf7554. Latest consensus state: current_term: 1 leader_uuid: "800904235b724421b218e3eb1fcf7554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.551040 27462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "800904235b724421b218e3eb1fcf7554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.551471 27460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:03:28.551498 27462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:03:28.585721 27480 catalog_manager.cc:1259] Loaded cluster ID: 04c766bafcb4436482c2b5d92a8dc624
I20260927 07:03:28.585923 27480 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f: loading cluster ID for follower catalog manager: success
I20260927 07:03:28.593986 27480 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f: acquiring CA information for follower catalog manager: success
I20260927 07:03:28.596828 27480 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7764c61f07ca489797fff7ed5d96889f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:03:28.915942 27417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "800904235b724421b218e3eb1fcf7554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800904235b724421b218e3eb1fcf7554" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 42257 } } peers { permanent_uuid: "7764c61f07ca489797fff7ed5d96889f" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 41665 } } peers { permanent_uuid: "1e0d58df9dbe447bad12aab145064a45" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 46401 } } }
I20260927 07:03:28.916285 27417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800904235b724421b218e3eb1fcf7554 [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:29.423655 27457 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45: acquiring CA information for follower catalog manager: success
I20260927 07:03:29.425606 27457 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e0d58df9dbe447bad12aab145064a45: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:03:29.624821 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 27195
I20260927 07:03:29.626063 27257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:29.743920 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 27195
I20260927 07:03:29.761904 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 27269
I20260927 07:03:29.763159 27331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:29.870478 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 27269
I20260927 07:03:29.887310 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 27343
I20260927 07:03:29.888425 27405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:29.995568 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 27343
2026-09-27T07:03:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4459 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:03:30 dist-test-slave-p6hz krb5kdc[27488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:03:30 dist-test-slave-p6hz krb5kdc[27488](info): set up 2 sockets
Sep 27 07:03:30 dist-test-slave-p6hz krb5kdc[27488](info): commencing operation
krb5kdc: starting...
W20260927 07:03:31.019631 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:03:31 dist-test-slave-p6hz krb5kdc[27488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:03:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:03:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:03:31.247030 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:44579
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:42873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:44579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:31.583374 27501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:31.583762 27501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:31.583895 27501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:31.591285 27501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:31.591403 27501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:31.591467 27501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:31.591547 27501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:31.591596 27501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:31.601580 27501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:42873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:44579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:44579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:31.602957 27501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:31.604318 27501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:31.615816 27506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:31.615864 27509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:31.615888 27507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:31.616874 27501 server_base.cc:1034] running on GCE node
I20260927 07:03:31.617648 27501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:31.619077 27501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:31.620332 27501 hybrid_clock.cc:648] HybridClock initialized: now 1790492611620265 us; error 52 us; skew 500 ppm
I20260927 07:03:31.620712 27501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:31 dist-test-slave-p6hz krb5kdc[27488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492611, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:31.623804 27501 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:03:31.625239 27501 webserver.cc:460] Webserver started at http://127.13.121.254:36683/ using document root <none> and password file <none>
I20260927 07:03:31.625888 27501 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:31.626003 27501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:31.626278 27501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:31.629244 27501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "12e55471c7cc40668d53acdfdf571f65"
format_stamp: "Formatted at 2026-09-27 07:03:31 on dist-test-slave-p6hz"
server_key: "f3b287069237fd04f472ebfe38705744"
server_key_iv: "2fa93c2fc2b04004fa3326791d033e46"
server_key_version: "encryptionkey@0"
I20260927 07:03:31.629982 27501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "12e55471c7cc40668d53acdfdf571f65"
format_stamp: "Formatted at 2026-09-27 07:03:31 on dist-test-slave-p6hz"
server_key: "f3b287069237fd04f472ebfe38705744"
server_key_iv: "2fa93c2fc2b04004fa3326791d033e46"
server_key_version: "encryptionkey@0"
I20260927 07:03:31.635121 27501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260927 07:03:31.638461 27516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:31.639832 27501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 07:03:31.639990 27501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "12e55471c7cc40668d53acdfdf571f65"
format_stamp: "Formatted at 2026-09-27 07:03:31 on dist-test-slave-p6hz"
server_key: "f3b287069237fd04f472ebfe38705744"
server_key_iv: "2fa93c2fc2b04004fa3326791d033e46"
server_key_version: "encryptionkey@0"
I20260927 07:03:31.640172 27501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:31.660382 27501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:31.664669 27501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:31.664933 27501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:31.681882 27501 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:44579
I20260927 07:03:31.681926 27567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:44579 every 8 connection(s)
I20260927 07:03:31.683235 27501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:03:31.686856 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:31.688637 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 27501
I20260927 07:03:31.688849 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:03:31.689700 13799 external_mini_cluster.cc:1428] Setting key d998ad2cb81dd72ede58c1d4125a7d6e
I20260927 07:03:31.698489 27568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65: Bootstrap starting.
I20260927 07:03:31.701495 27568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:31.702436 27568 log.cc:824] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65: Log is configured to *not* fsync() on all Append() calls
Sep 27 07:03:31 dist-test-slave-p6hz krb5kdc[27488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:31.705526 27568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65: No bootstrap required, opened a new log
I20260927 07:03:31.711979 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:31.691779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51888 (local address 127.13.121.254:44579)
0927 07:03:31.692562 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:31.692575 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:31.692618 (+ 43us) server_negotiation.cc:407] Connection header received
0927 07:03:31.694332 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:31.694358 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:31.694763 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:31.695160 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:31.696662 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:31.697945 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:31.699358 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:31.700048 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:31.700208 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:31.704463 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:31.704474 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:31.704491 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:31.704529 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:31.707504 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:31.708406 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:31.708415 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:31.708424 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:31.708611 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:31.709008 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:31.709012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:31.709014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:31.709597 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:31.709843 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:31.710403 (+ 560us) server_negotiation.cc:299] Negotiation successful
0927 07:03:31.710821 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":319,"threads_started":1}
I20260927 07:03:31.714066 27568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e55471c7cc40668d53acdfdf571f65" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44579 } }
I20260927 07:03:31.714422 27568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:31.714515 27568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e55471c7cc40668d53acdfdf571f65, State: Initialized, Role: FOLLOWER
I20260927 07:03:31.715126 27568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e55471c7cc40668d53acdfdf571f65" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44579 } }
I20260927 07:03:31.715404 27568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:03:31.715539 27568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:03:31.715706 27568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:31.715953 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:03:31.718164 27568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e55471c7cc40668d53acdfdf571f65" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44579 } }
I20260927 07:03:31.718784 27568 leader_election.cc:302] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12e55471c7cc40668d53acdfdf571f65; no voters:
I20260927 07:03:31.719389 27568 leader_election.cc:288] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:03:31.719537 27573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:03:31.720187 27573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [term 1 LEADER]: Becoming Leader. State: Replica: 12e55471c7cc40668d53acdfdf571f65, State: Running, Role: LEADER
I20260927 07:03:31.720746 27573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e55471c7cc40668d53acdfdf571f65" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44579 } }
I20260927 07:03:31.721424 27568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:31.723230 27575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12e55471c7cc40668d53acdfdf571f65. Latest consensus state: current_term: 1 leader_uuid: "12e55471c7cc40668d53acdfdf571f65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e55471c7cc40668d53acdfdf571f65" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44579 } } }
I20260927 07:03:31.723306 27574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12e55471c7cc40668d53acdfdf571f65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e55471c7cc40668d53acdfdf571f65" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 44579 } } }
I20260927 07:03:31.723744 27574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:31.723744 27575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65 [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:31.724638 27579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:03:31.731348 27579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:03:31.740175 27579 catalog_manager.cc:1347] Generated new cluster ID: 2203af5681eb4702abe599539dc450c3
I20260927 07:03:31.740302 27579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:03:31.764083 27579 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:03:31.766021 27579 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:03:31.785701 27579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12e55471c7cc40668d53acdfdf571f65: Generated new TSK 0
I20260927 07:03:31.786934 27579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:03:31.828692 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 27501
I20260927 07:03:31.829960 27563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:31.928416 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 27501
2026-09-27T07:03:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1929 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:03:32 dist-test-slave-p6hz krb5kdc[27596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:03:32 dist-test-slave-p6hz krb5kdc[27596](info): set up 2 sockets
Sep 27 07:03:32 dist-test-slave-p6hz krb5kdc[27596](info): commencing operation
krb5kdc: starting...
W20260927 07:03:32.943552 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:03:33 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:03:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:03:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:03:33.171504 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:34703
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:34703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:33.495935 27609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:33.496259 27609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:33.496392 27609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:33.503754 27609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:33.503880 27609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:33.503958 27609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:33.504035 27609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:33.504093 27609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:33.514129 27609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:34703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:34703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:33.515597 27609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:33.516911 27609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:33.528610 27617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:33.528642 27615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:33.528581 27614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:33.529322 27609 server_base.cc:1034] running on GCE node
I20260927 07:03:33.530107 27609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:33.531623 27609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:33.532860 27609 hybrid_clock.cc:648] HybridClock initialized: now 1790492613532796 us; error 45 us; skew 500 ppm
I20260927 07:03:33.533232 27609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:33 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:33.544009 27609 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:03:33.545584 27609 webserver.cc:460] Webserver started at http://127.13.121.254:41773/ using document root <none> and password file <none>
I20260927 07:03:33.546262 27609 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:33.546365 27609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:33.546627 27609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:33.549778 27609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "d6f9251dce064af48933aacb03ff9bbe"
format_stamp: "Formatted at 2026-09-27 07:03:33 on dist-test-slave-p6hz"
server_key: "617268d123fef3114df9ae6ebbc05a66"
server_key_iv: "81d79c74b71bc2a1e6d1e546a5b16d94"
server_key_version: "encryptionkey@0"
I20260927 07:03:33.550491 27609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "d6f9251dce064af48933aacb03ff9bbe"
format_stamp: "Formatted at 2026-09-27 07:03:33 on dist-test-slave-p6hz"
server_key: "617268d123fef3114df9ae6ebbc05a66"
server_key_iv: "81d79c74b71bc2a1e6d1e546a5b16d94"
server_key_version: "encryptionkey@0"
I20260927 07:03:33.555884 27609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260927 07:03:33.559608 27624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:33.561149 27609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260927 07:03:33.561316 27609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "d6f9251dce064af48933aacb03ff9bbe"
format_stamp: "Formatted at 2026-09-27 07:03:33 on dist-test-slave-p6hz"
server_key: "617268d123fef3114df9ae6ebbc05a66"
server_key_iv: "81d79c74b71bc2a1e6d1e546a5b16d94"
server_key_version: "encryptionkey@0"
I20260927 07:03:33.561507 27609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:33.596274 27609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:33.600793 27609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:33.601054 27609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:33.618841 27609 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:34703
I20260927 07:03:33.618875 27675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:34703 every 8 connection(s)
I20260927 07:03:33.620256 27609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:03:33.623224 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 27609
I20260927 07:03:33.623433 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:03:33.624094 13799 external_mini_cluster.cc:1428] Setting key 4b5842fb09d4d93b67d3844491ea704c
I20260927 07:03:33.624116 27676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:33.638340 27676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Bootstrap starting.
Sep 27 07:03:33 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:33.642277 27676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:33.643756 27676 log.cc:824] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:33.647298 27676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: No bootstrap required, opened a new log
I20260927 07:03:33.648377 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:33.625998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.13.121.254:34703)
0927 07:03:33.626804 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:33.626819 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:33.626872 (+ 53us) server_negotiation.cc:407] Connection header received
0927 07:03:33.628850 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:33.628882 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:33.629416 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:33.629891 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:33.631237 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:33.633156 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:33.634500 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:33.635608 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:33.635804 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:33.639771 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:33.639787 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:33.639808 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:33.639869 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:33.643720 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:33.644590 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:33.644597 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:33.644603 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:33.644776 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:33.645215 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:33.645220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:33.645223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:33.645988 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:33.646300 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:33.646610 (+ 310us) server_negotiation.cc:299] Negotiation successful
0927 07:03:33.647037 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":326,"threads_started":1}
I20260927 07:03:33.655128 27676 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:33.655540 27676 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:33.655622 27676 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6f9251dce064af48933aacb03ff9bbe, State: Initialized, Role: FOLLOWER
I20260927 07:03:33.656111 27676 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:33.656303 27676 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:03:33.656394 27676 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:03:33.656527 27676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:33.658964 27676 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:33.659493 27676 leader_election.cc:302] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6f9251dce064af48933aacb03ff9bbe; no voters:
I20260927 07:03:33.660115 27676 leader_election.cc:288] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:03:33.660274 27681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:03:33.660990 27681 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 1 LEADER]: Becoming Leader. State: Replica: d6f9251dce064af48933aacb03ff9bbe, State: Running, Role: LEADER
I20260927 07:03:33.661530 27681 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:33.662209 27676 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:33.664434 27682 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6f9251dce064af48933aacb03ff9bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } } }
I20260927 07:03:33.664968 27683 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6f9251dce064af48933aacb03ff9bbe. Latest consensus state: current_term: 1 leader_uuid: "d6f9251dce064af48933aacb03ff9bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } } }
I20260927 07:03:33.665325 27682 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:33.665429 27683 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:33.668533 27688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:03:33.673337 27688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:03:33.681907 27688 catalog_manager.cc:1347] Generated new cluster ID: 86936951dda545b78df036ebb65eeb84
I20260927 07:03:33.682029 27688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:03:33.724848 27688 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:03:33.726167 27688 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:03:33.735173 27688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Generated new TSK 0
I20260927 07:03:33.736042 27688 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:03:33.795172 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:34703
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:34.127271 27702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:34.127606 27702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:34.127709 27702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:34.135304 27702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:34.135418 27702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:34.135545 27702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:03:34.145552 27702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:34.146838 27702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:34.148288 27702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:34.161254 27708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:34.161293 27710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:34.161229 27707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:34.161991 27702 server_base.cc:1034] running on GCE node
I20260927 07:03:34.162639 27702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:34.163676 27702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:34.164889 27702 hybrid_clock.cc:648] HybridClock initialized: now 1790492614164826 us; error 54 us; skew 500 ppm
I20260927 07:03:34.165211 27702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:34 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492614, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:34.168603 27702 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:03:34.170213 27702 webserver.cc:460] Webserver started at http://127.13.121.193:36437/ using document root <none> and password file <none>
I20260927 07:03:34.170920 27702 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:34.171039 27702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:34.171370 27702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:34.174288 27702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "64abfceb9020451192fd2e81881aee93"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "5a59f31a75a9fed3438cea680d18ce3f"
server_key_iv: "6bef08e7e1f330ccc79b4f62fa9f4bb4"
server_key_version: "encryptionkey@0"
I20260927 07:03:34.175077 27702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "64abfceb9020451192fd2e81881aee93"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "5a59f31a75a9fed3438cea680d18ce3f"
server_key_iv: "6bef08e7e1f330ccc79b4f62fa9f4bb4"
server_key_version: "encryptionkey@0"
I20260927 07:03:34.180155 27702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260927 07:03:34.183574 27717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:34.184964 27702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:03:34.185123 27702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "64abfceb9020451192fd2e81881aee93"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "5a59f31a75a9fed3438cea680d18ce3f"
server_key_iv: "6bef08e7e1f330ccc79b4f62fa9f4bb4"
server_key_version: "encryptionkey@0"
I20260927 07:03:34.185313 27702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:34.212591 27702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:34.216998 27702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:34.217283 27702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:34.218273 27702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:34.219743 27702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:34.219822 27702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:34.219940 27702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:34.219993 27702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:34.250560 27702 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:37335
I20260927 07:03:34.250617 27829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:37335 every 8 connection(s)
I20260927 07:03:34.251868 27702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:03:34.256963 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 27702
I20260927 07:03:34.257156 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:03:34.257781 13799 external_mini_cluster.cc:1428] Setting key 7073d9305f83d4f969a6c0422732e415
Sep 27 07:03:34 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492614, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:34.272481 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:34.254277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:46293 (local address 127.13.121.254:34703)
0927 07:03:34.254723 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:34.254728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:34.255652 (+ 924us) server_negotiation.cc:407] Connection header received
0927 07:03:34.257472 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:34.257478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:34.257571 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:34.257669 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:34.259308 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:34.260210 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:34.261833 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:34.262389 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:34.262499 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:34.266640 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:34.266646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:34.266650 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:34.266693 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:34.269209 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:34.270141 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:34.270145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:34.270147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:34.270276 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:34.270726 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:34.270729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:34.270731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:34.271007 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:34.271120 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:34.272019 (+ 899us) server_negotiation.cc:299] Negotiation successful
0927 07:03:34.272212 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":207,"threads_started":1}
I20260927 07:03:34.273794 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:34.254803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:46293)
0927 07:03:34.255504 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:34.255546 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:34.257203 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:34.257933 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:34.257943 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:34.258526 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:34.259109 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:34.259126 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:34.260368 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:34.260372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:34.261677 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:34.261690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:34.262651 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:34.262657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:34.262956 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:34.263745 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:34.263775 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:34.266360 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:34.269375 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:34.269384 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:34.269429 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:34.270023 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:34.270371 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:34.270376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:34.270380 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:34.270619 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:34.271128 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:34.271164 (+ 36us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:34.271624 (+ 460us) client_negotiation.cc:769] Sending connection context
0927 07:03:34.271927 (+ 303us) client_negotiation.cc:240] Negotiation successful
0927 07:03:34.272477 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":279,"threads_started":1}
I20260927 07:03:34.275434 27830 heartbeater.cc:347] Connected to a master server at 127.13.121.254:34703
I20260927 07:03:34.275734 27830 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:34.276473 27830 heartbeater.cc:511] Master 127.13.121.254:34703 requested a full tablet report, sending...
I20260927 07:03:34.278594 27641 ts_manager.cc:194] Registered new tserver with Master: 64abfceb9020451192fd2e81881aee93 (127.13.121.193:37335)
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:03:34.280143 27641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:46293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:03:34.306946 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:34703
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:34.633000 27836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:34.633378 27836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:34.633526 27836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:34.644064 27836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:34.644187 27836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:34.644301 27836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:03:34.654140 27836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:34.655422 27836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:34.656693 27836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:34.669617 27841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:34.669698 27844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:34.669668 27842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:34.670428 27836 server_base.cc:1034] running on GCE node
I20260927 07:03:34.671053 27836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:34.671995 27836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:34.673187 27836 hybrid_clock.cc:648] HybridClock initialized: now 1790492614673122 us; error 60 us; skew 500 ppm
I20260927 07:03:34.673483 27836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:34 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492614, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:34.676527 27836 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:03:34.677986 27836 webserver.cc:460] Webserver started at http://127.13.121.194:35007/ using document root <none> and password file <none>
I20260927 07:03:34.678649 27836 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:34.678768 27836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:34.679077 27836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:34.682045 27836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "c9be399be7f1481a8131347c2ab2fa8b"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "75e9cf1cb8a79b83c8b1a681d32c80a6"
server_key_iv: "6c9c7dd8d6cf9ad50e7f7993fb028a15"
server_key_version: "encryptionkey@0"
I20260927 07:03:34.682847 27836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "c9be399be7f1481a8131347c2ab2fa8b"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "75e9cf1cb8a79b83c8b1a681d32c80a6"
server_key_iv: "6c9c7dd8d6cf9ad50e7f7993fb028a15"
server_key_version: "encryptionkey@0"
I20260927 07:03:34.688122 27836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:03:34.692364 27851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:34.693826 27836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:03:34.694005 27836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "c9be399be7f1481a8131347c2ab2fa8b"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "75e9cf1cb8a79b83c8b1a681d32c80a6"
server_key_iv: "6c9c7dd8d6cf9ad50e7f7993fb028a15"
server_key_version: "encryptionkey@0"
I20260927 07:03:34.694206 27836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:34.733749 27836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:34.738469 27836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:34.738775 27836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:34.739886 27836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:34.741339 27836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:34.741415 27836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:34.741524 27836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:34.741578 27836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:34.772951 27836 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:38137
I20260927 07:03:34.772986 27963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:38137 every 8 connection(s)
I20260927 07:03:34.774315 27836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:03:34.778460 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 27836
I20260927 07:03:34.778647 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:03:34.779100 13799 external_mini_cluster.cc:1428] Setting key 5fc3e536928db1a9e29b8cabf906aa8c
Sep 27 07:03:34 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492614, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:34.795042 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:34.777018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:46595 (local address 127.13.121.254:34703)
0927 07:03:34.777466 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:34.777470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:34.778398 (+ 928us) server_negotiation.cc:407] Connection header received
0927 07:03:34.780400 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:34.780405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:34.780475 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:34.780567 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:34.782090 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:34.782791 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:34.784415 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:34.784842 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:34.784958 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:34.788717 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:34.788723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:34.788726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:34.788763 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:34.791630 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:34.792449 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:34.792455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:34.792459 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:34.792596 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:34.793028 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:34.793032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:34.793034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:34.793310 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:34.793449 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:34.794202 (+ 753us) server_negotiation.cc:299] Negotiation successful
0927 07:03:34.794467 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":210,"threads_started":1}
I20260927 07:03:34.796110 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:34.777580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:46595)
0927 07:03:34.778244 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:34.778285 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:34.780116 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:34.780834 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:34.780842 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:34.781348 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:34.781913 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:34.781931 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:34.782942 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:34.782946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:34.784264 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:34.784277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:34.785069 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:34.785077 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:34.785303 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:34.785990 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:34.786013 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:34.788399 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:34.791804 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:34.791815 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:34.791834 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:34.792304 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:34.792712 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:34.792717 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:34.792721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:34.792911 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:34.793455 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:34.793464 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:34.793823 (+ 359us) client_negotiation.cc:769] Sending connection context
0927 07:03:34.794077 (+ 254us) client_negotiation.cc:240] Negotiation successful
0927 07:03:34.794666 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":262,"threads_started":1}
I20260927 07:03:34.797562 27964 heartbeater.cc:347] Connected to a master server at 127.13.121.254:34703
I20260927 07:03:34.797878 27964 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:03:34.798710 27964 heartbeater.cc:511] Master 127.13.121.254:34703 requested a full tablet report, sending...
I20260927 07:03:34.800266 27641 ts_manager.cc:194] Registered new tserver with Master: c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137)
I20260927 07:03:34.801163 27641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:46595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:03:34.830386 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:34703
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:35.152189 27970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:35.152498 27970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:35.152595 27970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:35.160022 27970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:35.160202 27970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:35.160343 27970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:03:35.170677 27970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:35.171958 27970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:35.173237 27970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:35.185974 27975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:35.186024 27978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:35.186132 27976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:35.186730 27970 server_base.cc:1034] running on GCE node
I20260927 07:03:35.187403 27970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:35.188340 27970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:35.189546 27970 hybrid_clock.cc:648] HybridClock initialized: now 1790492615189495 us; error 43 us; skew 500 ppm
I20260927 07:03:35.189881 27970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:35 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492615, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:35.193080 27970 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:03:35.194500 27970 webserver.cc:460] Webserver started at http://127.13.121.195:40763/ using document root <none> and password file <none>
I20260927 07:03:35.195379 27970 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:35.195501 27970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:35.195775 27970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:35.198686 27970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "79c1aac6ee35417f82cc6bada1eae949"
format_stamp: "Formatted at 2026-09-27 07:03:35 on dist-test-slave-p6hz"
server_key: "c03655822e0b45c19b3330b978b7bdfe"
server_key_iv: "9b548864d763a8ef893cfc83d4b5597a"
server_key_version: "encryptionkey@0"
I20260927 07:03:35.199517 27970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "79c1aac6ee35417f82cc6bada1eae949"
format_stamp: "Formatted at 2026-09-27 07:03:35 on dist-test-slave-p6hz"
server_key: "c03655822e0b45c19b3330b978b7bdfe"
server_key_iv: "9b548864d763a8ef893cfc83d4b5597a"
server_key_version: "encryptionkey@0"
I20260927 07:03:35.204978 27970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260927 07:03:35.209012 27985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:35.210366 27970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:03:35.210515 27970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "79c1aac6ee35417f82cc6bada1eae949"
format_stamp: "Formatted at 2026-09-27 07:03:35 on dist-test-slave-p6hz"
server_key: "c03655822e0b45c19b3330b978b7bdfe"
server_key_iv: "9b548864d763a8ef893cfc83d4b5597a"
server_key_version: "encryptionkey@0"
I20260927 07:03:35.210708 27970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:35.233363 27970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:35.237501 27970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:35.237771 27970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:35.238749 27970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:35.240144 27970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:35.240229 27970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:35.240399 27970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:35.240449 27970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:35.273321 27970 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:33893
I20260927 07:03:35.273370 28097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:33893 every 8 connection(s)
I20260927 07:03:35.274582 27970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:03:35.282464 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 27970
I20260927 07:03:35.282619 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:03:35.283113 13799 external_mini_cluster.cc:1428] Setting key ea1c7fa804216febb1191a93529d97d4
I20260927 07:03:35.283262 27830 heartbeater.cc:503] Master 127.13.121.254:34703 was elected leader, sending a full tablet report...
Sep 27 07:03:35 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492615, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:35.294620 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.277116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:42079 (local address 127.13.121.254:34703)
0927 07:03:35.277307 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.277311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.278415 (+ 1104us) server_negotiation.cc:407] Connection header received
0927 07:03:35.280292 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.280297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.280375 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.280463 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:35.281955 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.282834 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.285100 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.285592 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.285686 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.289386 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:35.289391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:35.289395 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:35.289432 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:35.291868 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.292603 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.292607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.292609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.292729 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.293107 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.293111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.293113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.293380 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:35.293513 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.294181 (+ 668us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.294357 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 07:03:35.295660 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.277548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:42079)
0927 07:03:35.278246 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:35.278285 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:35.280018 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:35.280684 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:35.280693 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:35.281209 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:35.281773 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.281796 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.283002 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.283008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.284933 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.284948 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.285825 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.285834 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.286135 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.286792 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:35.286816 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:35.289107 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:35.292014 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:35.292021 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:35.292035 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:35.292468 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:35.292812 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:35.292815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:35.292817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:35.292998 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:35.293482 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:35.293489 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:35.293837 (+ 348us) client_negotiation.cc:769] Sending connection context
0927 07:03:35.294068 (+ 231us) client_negotiation.cc:240] Negotiation successful
0927 07:03:35.294470 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":255,"threads_started":1}
I20260927 07:03:35.297179 28098 heartbeater.cc:347] Connected to a master server at 127.13.121.254:34703
I20260927 07:03:35.297515 28098 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:35.298498 28098 heartbeater.cc:511] Master 127.13.121.254:34703 requested a full tablet report, sending...
I20260927 07:03:35.299957 27641 ts_manager.cc:194] Registered new tserver with Master: 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893)
I20260927 07:03:35.300786 27641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:42079
I20260927 07:03:35.311002 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:03:35.327683 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.315341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.13.121.254:34703)
0927 07:03:35.315507 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.315513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.315897 (+ 384us) server_negotiation.cc:407] Connection header received
0927 07:03:35.316071 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.316075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.316156 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.316245 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:35.317542 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.318373 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.319699 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.320132 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.320234 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.322559 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:35.322565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:35.322568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:35.322607 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:35.325141 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.325761 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.325769 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.325771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.325892 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.326160 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.326163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.326165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.326470 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:35.326744 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.327070 (+ 326us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.327312 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 07:03:35.331561 27641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:03:35.338241 27641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:03:35.360558 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.351920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.13.121.195:33893)
0927 07:03:35.353380 (+ 1460us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.353387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.353412 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:03:35.353487 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.353492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.353651 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.353801 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:35.354532 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.355934 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.357885 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.359344 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.359548 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.359739 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.360042 (+ 303us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.360214 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1326,"thread_start_us":1139,"threads_started":1}
I20260927 07:03:35.360642 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.350579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.0.0.1:35860)
0927 07:03:35.351314 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:35.351354 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:35.351528 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:35.352361 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:35.352369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:35.352395 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:35.352814 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.352826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.354811 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.354816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.357477 (+ 2661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.357492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.359647 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.359654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.359836 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.359862 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 07:03:35.359940 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 07:03:35.360159 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":27,"thread_start_us":291,"threads_started":1}
I20260927 07:03:35.360678 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.351430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33893 (local address 127.0.0.1:52098)
0927 07:03:35.352071 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:35.352087 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:35.352195 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:35.353813 (+ 1618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:35.353817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:35.353840 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:35.354292 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.354305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.356064 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.356068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.357703 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.357719 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.359561 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.359571 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.359789 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.359840 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 07:03:35.359986 (+ 146us) client_negotiation.cc:240] Negotiation successful
0927 07:03:35.360164 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":91,"thread_start_us":1237,"threads_started":1}
I20260927 07:03:35.361948 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.350922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.13.121.193:37335)
0927 07:03:35.351598 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.351606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.351637 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:03:35.351728 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.351735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.351983 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.352166 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:35.352975 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.354668 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.357647 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.359388 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.361181 (+ 1793us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.361387 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.361470 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.361636 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":359,"threads_started":1}
I20260927 07:03:35.364123 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.352079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:38137 (local address 127.0.0.1:53974)
0927 07:03:35.354626 (+ 2547us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:35.354644 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:35.354831 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:35.355627 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:35.355635 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:35.355661 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:35.356051 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.356064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.358284 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.358288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.360043 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.360056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.363318 (+ 3262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.363322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.363527 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.363562 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:03:35.363648 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 07:03:35.363785 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2419,"thread_start_us":2264,"threads_started":1}
I20260927 07:03:35.364163 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.352536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53974 (local address 127.13.121.194:38137)
0927 07:03:35.354157 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.354165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.354726 (+ 561us) server_negotiation.cc:407] Connection header received
0927 07:03:35.355040 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.355048 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.355276 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.355418 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:35.356302 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.358122 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.360200 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.361936 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.362983 (+ 1047us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.363126 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.363707 (+ 581us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.363885 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":293,"threads_started":1}
I20260927 07:03:35.365437 28033 tablet_service.cc:1467] Processing CreateTablet for tablet 426da4bfb71b4b43ad61919269a2d06f (DEFAULT_TABLE table=test-table [id=fd730f18b0cc4bf29704470e7466333d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:35.365862 27765 tablet_service.cc:1467] Processing CreateTablet for tablet 426da4bfb71b4b43ad61919269a2d06f (DEFAULT_TABLE table=test-table [id=fd730f18b0cc4bf29704470e7466333d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:35.367223 27765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 426da4bfb71b4b43ad61919269a2d06f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:35.367228 28033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 426da4bfb71b4b43ad61919269a2d06f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:35.367648 27899 tablet_service.cc:1467] Processing CreateTablet for tablet 426da4bfb71b4b43ad61919269a2d06f (DEFAULT_TABLE table=test-table [id=fd730f18b0cc4bf29704470e7466333d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:35.368737 27899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 426da4bfb71b4b43ad61919269a2d06f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:35.380779 28116 tablet_bootstrap.cc:492] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: Bootstrap starting.
I20260927 07:03:35.383874 28118 tablet_bootstrap.cc:492] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: Bootstrap starting.
I20260927 07:03:35.385277 28116 tablet_bootstrap.cc:654] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:35.385494 28117 tablet_bootstrap.cc:492] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: Bootstrap starting.
I20260927 07:03:35.386729 28116 log.cc:824] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:35.387391 28118 tablet_bootstrap.cc:654] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:35.388440 28118 log.cc:824] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:35.388696 28117 tablet_bootstrap.cc:654] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:35.389793 28116 tablet_bootstrap.cc:492] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: No bootstrap required, opened a new log
I20260927 07:03:35.389874 28117 log.cc:824] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:35.390059 28116 ts_tablet_manager.cc:1397] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.002s
I20260927 07:03:35.391414 28118 tablet_bootstrap.cc:492] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: No bootstrap required, opened a new log
I20260927 07:03:35.391685 28118 ts_tablet_manager.cc:1397] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260927 07:03:35.392768 28117 tablet_bootstrap.cc:492] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: No bootstrap required, opened a new log
I20260927 07:03:35.393072 28117 ts_tablet_manager.cc:1397] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260927 07:03:35.397951 28116 raft_consensus.cc:348] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } }
I20260927 07:03:35.398355 28116 raft_consensus.cc:374] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:35.398509 28116 raft_consensus.cc:729] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9be399be7f1481a8131347c2ab2fa8b, State: Initialized, Role: FOLLOWER
I20260927 07:03:35.399083 28116 consensus_queue.cc:260] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } }
I20260927 07:03:35.400162 27964 heartbeater.cc:503] Master 127.13.121.254:34703 was elected leader, sending a full tablet report...
I20260927 07:03:35.400872 28116 ts_tablet_manager.cc:1428] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260927 07:03:35.401926 28118 raft_consensus.cc:348] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } }
I20260927 07:03:35.402392 28118 raft_consensus.cc:374] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:35.402514 28118 raft_consensus.cc:729] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c1aac6ee35417f82cc6bada1eae949, State: Initialized, Role: FOLLOWER
I20260927 07:03:35.403245 28118 consensus_queue.cc:260] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } }
I20260927 07:03:35.403257 28117 raft_consensus.cc:348] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } }
I20260927 07:03:35.403685 28117 raft_consensus.cc:374] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:35.403810 28117 raft_consensus.cc:729] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64abfceb9020451192fd2e81881aee93, State: Initialized, Role: FOLLOWER
I20260927 07:03:35.404538 28098 heartbeater.cc:503] Master 127.13.121.254:34703 was elected leader, sending a full tablet report...
I20260927 07:03:35.404465 28117 consensus_queue.cc:260] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } }
I20260927 07:03:35.405256 28118 ts_tablet_manager.cc:1428] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260927 07:03:35.407347 28117 ts_tablet_manager.cc:1428] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260927 07:03:35.438791 28124 raft_consensus.cc:482] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:35.439070 28124 raft_consensus.cc:504] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } }
I20260927 07:03:35.440747 28124 leader_election.cc:288] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137), 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893)
I20260927 07:03:35.447327 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.441226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:41161 (local address 127.13.121.194:38137)
0927 07:03:35.441454 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.441461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.442142 (+ 681us) server_negotiation.cc:407] Connection header received
0927 07:03:35.442390 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.442396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.442478 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.442584 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:35.443119 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.443922 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.445649 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.446422 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.446575 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.446651 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.446923 (+ 272us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.447050 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 07:03:35.447398 28125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.441073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:38137 (local address 127.13.121.193:41161)
0927 07:03:35.442000 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:35.442024 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:35.442206 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:35.442615 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:35.442618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:35.442642 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:35.442960 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.442976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.444059 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.444062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.445490 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.445503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.446623 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.446629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.446775 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.446799 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 07:03:35.446878 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 07:03:35.447030 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":1201,"threads_started":1}
I20260927 07:03:35.447656 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.441191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:43689 (local address 127.13.121.195:33893)
0927 07:03:35.441396 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.441403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.442796 (+ 1393us) server_negotiation.cc:407] Connection header received
0927 07:03:35.442976 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.442980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.443043 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.443113 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:35.443649 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.444355 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.446002 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.446765 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.446906 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.446977 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.447292 (+ 315us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.447430 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 07:03:35.447783 28126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.441081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33893 (local address 127.13.121.193:43689)
0927 07:03:35.442672 (+ 1591us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:35.442699 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:35.442844 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:35.443175 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:35.443180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:35.443203 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:35.443499 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.443507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.444486 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.444491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.445851 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:35.445862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.446970 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.446976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:35.447116 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.447167 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 07:03:35.447251 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 07:03:35.447377 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1466,"mutex_wait_us":29,"thread_start_us":482,"threads_started":1}
I20260927 07:03:35.448747 27919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426da4bfb71b4b43ad61919269a2d06f" candidate_uuid: "64abfceb9020451192fd2e81881aee93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9be399be7f1481a8131347c2ab2fa8b" is_pre_election: true
I20260927 07:03:35.448747 28053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426da4bfb71b4b43ad61919269a2d06f" candidate_uuid: "64abfceb9020451192fd2e81881aee93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c1aac6ee35417f82cc6bada1eae949" is_pre_election: true
I20260927 07:03:35.449215 27919 raft_consensus.cc:2454] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64abfceb9020451192fd2e81881aee93 in term 0.
I20260927 07:03:35.449214 28053 raft_consensus.cc:2454] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64abfceb9020451192fd2e81881aee93 in term 0.
I20260927 07:03:35.449819 27718 leader_election.cc:302] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64abfceb9020451192fd2e81881aee93, c9be399be7f1481a8131347c2ab2fa8b; no voters:
I20260927 07:03:35.450196 28124 raft_consensus.cc:2781] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:03:35.450344 28124 raft_consensus.cc:482] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:03:35.450451 28124 raft_consensus.cc:3037] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:35.452658 28124 raft_consensus.cc:504] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } }
I20260927 07:03:35.453305 28124 leader_election.cc:288] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 1 election: Requested vote from peers c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137), 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893)
I20260927 07:03:35.453845 27919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426da4bfb71b4b43ad61919269a2d06f" candidate_uuid: "64abfceb9020451192fd2e81881aee93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9be399be7f1481a8131347c2ab2fa8b"
I20260927 07:03:35.453894 28053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426da4bfb71b4b43ad61919269a2d06f" candidate_uuid: "64abfceb9020451192fd2e81881aee93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c1aac6ee35417f82cc6bada1eae949"
I20260927 07:03:35.454099 28053 raft_consensus.cc:3037] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:35.454095 27919 raft_consensus.cc:3037] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:35.457532 27919 raft_consensus.cc:2454] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64abfceb9020451192fd2e81881aee93 in term 1.
I20260927 07:03:35.457532 28053 raft_consensus.cc:2454] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64abfceb9020451192fd2e81881aee93 in term 1.
I20260927 07:03:35.458114 27718 leader_election.cc:302] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64abfceb9020451192fd2e81881aee93, c9be399be7f1481a8131347c2ab2fa8b; no voters:
I20260927 07:03:35.458467 28124 raft_consensus.cc:2781] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:03:35.458949 28124 raft_consensus.cc:686] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 1 LEADER]: Becoming Leader. State: Replica: 64abfceb9020451192fd2e81881aee93, State: Running, Role: LEADER
I20260927 07:03:35.459491 28124 consensus_queue.cc:237] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } }
I20260927 07:03:35.464427 27640 catalog_manager.cc:5495] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64abfceb9020451192fd2e81881aee93 (127.13.121.193). New cstate: current_term: 1 leader_uuid: "64abfceb9020451192fd2e81881aee93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } health_report { overall_health: UNKNOWN } } }
I20260927 07:03:35.491369 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.485785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.13.121.193:37335)
0927 07:03:35.485980 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.485984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.486001 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:03:35.486093 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.486097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.486213 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.486377 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:03:35.486856 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.487762 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.489210 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.489707 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.489814 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.490325 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:03:35.490826 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:03:35.490938 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.491040 (+ 102us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.491110 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:03:35.499966 28053 raft_consensus.cc:1261] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [term 1 FOLLOWER]: Refusing update from remote peer 64abfceb9020451192fd2e81881aee93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:35.499969 27919 raft_consensus.cc:1261] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [term 1 FOLLOWER]: Refusing update from remote peer 64abfceb9020451192fd2e81881aee93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:35.501025 28124 consensus_queue.cc:1035] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:03:35.501336 28127 consensus_queue.cc:1035] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 07:03:35.504057 27831 tablet.cc:2369] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:03:35.514328 28130 mvcc.cc:205] Tried to move back new op lower bound from 7333857753077534720 to 7333857752925786112. Current Snapshot: MvccSnapshot[applied={T|T < 7333857753077534720}]
I20260927 07:03:35.529357 27641 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:03:35.529667 27641 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:03:35.533815 27641 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Sending DeleteTablet for 3 replicas of tablet 426da4bfb71b4b43ad61919269a2d06f
I20260927 07:03:35.535218 28033 tablet_service.cc:1514] Processing DeleteTablet for tablet 426da4bfb71b4b43ad61919269a2d06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:35 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:52098
I20260927 07:03:35.535260 27765 tablet_service.cc:1514] Processing DeleteTablet for tablet 426da4bfb71b4b43ad61919269a2d06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:35 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35860
I20260927 07:03:35.535986 27899 tablet_service.cc:1514] Processing DeleteTablet for tablet 426da4bfb71b4b43ad61919269a2d06f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:35 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:53974
I20260927 07:03:35.536312 28140 tablet_replica.cc:330] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: stopping tablet replica
I20260927 07:03:35.536469 28141 tablet_replica.cc:330] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: stopping tablet replica
I20260927 07:03:35.536762 28140 raft_consensus.cc:2229] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:03:35.536928 28141 raft_consensus.cc:2229] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:03:35.537104 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 27702
I20260927 07:03:35.537364 28140 raft_consensus.cc:2258] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:35.537634 28141 raft_consensus.cc:2258] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:35.539865 28140 ts_tablet_manager.cc:1905] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:35.546196 28141 ts_tablet_manager.cc:1905] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:35.547766 28140 ts_tablet_manager.cc:1918] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:35.548079 28140 log.cc:1196] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/426da4bfb71b4b43ad61919269a2d06f
I20260927 07:03:35.548710 28140 ts_tablet_manager.cc:1939] T 426da4bfb71b4b43ad61919269a2d06f P 79c1aac6ee35417f82cc6bada1eae949: Deleting consensus metadata
I20260927 07:03:35.551565 27628 catalog_manager.cc:4841] TS 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893): tablet 426da4bfb71b4b43ad61919269a2d06f (table test-table [id=fd730f18b0cc4bf29704470e7466333d]) successfully deleted
I20260927 07:03:35.553417 28142 tablet_replica.cc:330] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: stopping tablet replica
I20260927 07:03:35.553905 28142 raft_consensus.cc:2229] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:03:35.554535 28142 raft_consensus.cc:2258] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:35.557083 28142 ts_tablet_manager.cc:1905] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:35.558647 28141 ts_tablet_manager.cc:1918] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:35.558876 28141 log.cc:1196] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/426da4bfb71b4b43ad61919269a2d06f
I20260927 07:03:35.559604 28141 ts_tablet_manager.cc:1939] T 426da4bfb71b4b43ad61919269a2d06f P 64abfceb9020451192fd2e81881aee93: Deleting consensus metadata
I20260927 07:03:35.561647 27625 catalog_manager.cc:4841] TS 64abfceb9020451192fd2e81881aee93 (127.13.121.193:37335): tablet 426da4bfb71b4b43ad61919269a2d06f (table test-table [id=fd730f18b0cc4bf29704470e7466333d]) successfully deleted
Sep 27 07:03:35 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:03:35.564563 28142 ts_tablet_manager.cc:1918] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:35.564745 28142 log.cc:1196] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/426da4bfb71b4b43ad61919269a2d06f
I20260927 07:03:35.565284 28142 ts_tablet_manager.cc:1939] T 426da4bfb71b4b43ad61919269a2d06f P c9be399be7f1481a8131347c2ab2fa8b: Deleting consensus metadata
I20260927 07:03:35.567284 27625 catalog_manager.cc:4841] TS c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137): tablet 426da4bfb71b4b43ad61919269a2d06f (table test-table [id=fd730f18b0cc4bf29704470e7466333d]) successfully deleted
I20260927 07:03:35.570289 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.537897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.13.121.193:37335)
0927 07:03:35.538092 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.538098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.550035 (+ 11937us) server_negotiation.cc:407] Connection header received
0927 07:03:35.550260 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.550267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.550378 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.550491 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:35.553895 (+ 3404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.555118 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.557177 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.558013 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.558125 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.563821 (+ 5696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:35.563836 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:35.563851 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:35.563891 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:35.567198 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.568027 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.568036 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.568038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.568182 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.568665 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.568669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.568675 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.569073 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:35.569304 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.569749 (+ 445us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.569946 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:03:35.570987 27825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:35.669479 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 27702
I20260927 07:03:35.685304 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 27836
Sep 27 07:03:35 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:03:35.700786 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.686100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53982 (local address 127.13.121.194:38137)
0927 07:03:35.686284 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.686289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.686306 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:03:35.686423 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.686427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.686488 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.686565 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:35.687736 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.688494 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.689938 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.690377 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.690481 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.694712 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:35.694727 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:35.694743 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:35.694784 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:35.697707 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.698552 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.698562 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.698565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.698818 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.699240 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.699245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.699248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.699767 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:35.700005 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.700260 (+ 255us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.700434 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:03:35.701588 27959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:35.792249 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 27836
I20260927 07:03:35.806489 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 27970
Sep 27 07:03:35 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:03:35.821588 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:35.807286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52100 (local address 127.13.121.195:33893)
0927 07:03:35.807460 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:35.807466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:35.807514 (+ 48us) server_negotiation.cc:407] Connection header received
0927 07:03:35.807713 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:35.807718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:35.807788 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:35.807883 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:35.809157 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.809887 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.811165 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:35.811622 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:35.811737 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:35.816131 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:35.816148 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:35.816166 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:35.816210 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:35.818758 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.819636 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.819644 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.819647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.819814 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:35.820239 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:35.820242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:35.820244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:35.820598 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:35.820831 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:35.821155 (+ 324us) server_negotiation.cc:299] Negotiation successful
0927 07:03:35.821345 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 07:03:35.822212 28093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:35.910465 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 27970
I20260927 07:03:35.924090 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 27609
I20260927 07:03:35.925238 27671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:36.034569 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 27609
I20260927 07:03:36.052335 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:34703
--webserver_interface=127.13.121.254
--webserver_port=41773
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:34703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:36.379119 28148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:36.379493 28148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:36.379631 28148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:36.387029 28148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:36.387181 28148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:36.387243 28148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:36.387319 28148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:36.387380 28148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:36.397599 28148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:34703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:34703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=41773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:36.398926 28148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:36.400288 28148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:36.422183 28155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:36.422302 28154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:36.422194 28157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:36.423072 28148 server_base.cc:1034] running on GCE node
I20260927 07:03:36.424080 28148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:36.425840 28148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:36.427088 28148 hybrid_clock.cc:648] HybridClock initialized: now 1790492616427029 us; error 47 us; skew 500 ppm
I20260927 07:03:36.427583 28148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:36 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:36.431121 28148 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:03:36.432932 28148 webserver.cc:460] Webserver started at http://127.13.121.254:41773/ using document root <none> and password file <none>
I20260927 07:03:36.433689 28148 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:36.433805 28148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:36.439972 28148 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260927 07:03:36.443126 28164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:36.444783 28148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:03:36.444993 28148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "d6f9251dce064af48933aacb03ff9bbe"
format_stamp: "Formatted at 2026-09-27 07:03:33 on dist-test-slave-p6hz"
server_key: "617268d123fef3114df9ae6ebbc05a66"
server_key_iv: "81d79c74b71bc2a1e6d1e546a5b16d94"
server_key_version: "encryptionkey@0"
I20260927 07:03:36.446171 28148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:36.463819 28148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:36.468482 28148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:36.468762 28148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:36.486718 28148 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:34703
I20260927 07:03:36.486739 28215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:34703 every 8 connection(s)
I20260927 07:03:36.488384 28148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:03:36.493793 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 28148
I20260927 07:03:36.494766 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:37335
--local_ip_for_outbound_sockets=127.13.121.193
--tserver_master_addrs=127.13.121.254:34703
--webserver_port=36437
--webserver_interface=127.13.121.193
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:03:36.496608 28216 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:03:36.501511 28216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Bootstrap starting.
I20260927 07:03:36.522248 28216 log.cc:824] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:36.535506 28216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:03:36.536056 28216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Bootstrap complete.
I20260927 07:03:36.545559 28216 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:36.546434 28216 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6f9251dce064af48933aacb03ff9bbe, State: Initialized, Role: FOLLOWER
I20260927 07:03:36.547075 28216 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:36.547361 28216 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:03:36.547487 28216 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:03:36.547648 28216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:03:36.551865 28216 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:36.552394 28216 leader_election.cc:302] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6f9251dce064af48933aacb03ff9bbe; no voters:
I20260927 07:03:36.552981 28216 leader_election.cc:288] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 07:03:36.553126 28220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 2 FOLLOWER]: Leader election won for term 2
I20260927 07:03:36.554247 28220 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 2 LEADER]: Becoming Leader. State: Replica: d6f9251dce064af48933aacb03ff9bbe, State: Running, Role: LEADER
I20260927 07:03:36.554900 28220 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:36.555245 28216 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:36.557348 28221 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d6f9251dce064af48933aacb03ff9bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } } }
I20260927 07:03:36.557723 28221 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:36.559957 28222 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6f9251dce064af48933aacb03ff9bbe. Latest consensus state: current_term: 2 leader_uuid: "d6f9251dce064af48933aacb03ff9bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } } }
I20260927 07:03:36.560248 28222 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:36.561388 28226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:03:36.570711 28226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:03:36.572527 28226 catalog_manager.cc:1259] Loaded cluster ID: 86936951dda545b78df036ebb65eeb84
I20260927 07:03:36.572749 28226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:03:36.576609 28226 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260927 07:03:36.595940 28226 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260927 07:03:36.606462 28226 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:03:36.608578 28226 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Loaded TSK: 0
I20260927 07:03:36.610445 28226 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 07:03:36.864049 28218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:36.864331 28218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:36.864444 28218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:36.871908 28218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:36.872028 28218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:36.872192 28218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:03:36.882257 28218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:37335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=36437
--tserver_master_addrs=127.13.121.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:36.883622 28218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:36.884951 28218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:36.899636 28244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:36.900292 28243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:36.903513 28246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:36.905040 28218 server_base.cc:1034] running on GCE node
I20260927 07:03:36.905755 28218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:36.906847 28218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:36.908072 28218 hybrid_clock.cc:648] HybridClock initialized: now 1790492616908029 us; error 38 us; skew 500 ppm
I20260927 07:03:36.908422 28218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:36 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:36.911724 28218 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:03:36.913476 28218 webserver.cc:460] Webserver started at http://127.13.121.193:36437/ using document root <none> and password file <none>
I20260927 07:03:36.914218 28218 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:36.914350 28218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:36.920147 28218 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260927 07:03:36.923230 28253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:36.924789 28218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:03:36.924950 28218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "64abfceb9020451192fd2e81881aee93"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "5a59f31a75a9fed3438cea680d18ce3f"
server_key_iv: "6bef08e7e1f330ccc79b4f62fa9f4bb4"
server_key_version: "encryptionkey@0"
I20260927 07:03:36.925951 28218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:36.938208 28218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:36.942215 28218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:36.942548 28218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:36.943542 28218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:36.944972 28218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:36.945066 28218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:36.945170 28218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:36.945237 28218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:36.977789 28218 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:37335
I20260927 07:03:36.977849 28365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:37335 every 8 connection(s)
I20260927 07:03:36.979295 28218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:03:36.979794 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 28218
I20260927 07:03:36.980535 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:38137
--local_ip_for_outbound_sockets=127.13.121.194
--tserver_master_addrs=127.13.121.254:34703
--webserver_port=35007
--webserver_interface=127.13.121.194
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:03:36 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492616, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:37.006193 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:36.982794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:54695)
0927 07:03:36.983730 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:36.983783 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:36.986034 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:36.987680 (+ 1646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:36.987692 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:36.988257 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:36.988852 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:36.988872 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:36.991161 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:36.991168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:36.992477 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:36.992491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:36.993856 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:36.993861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:36.994062 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:36.994737 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:36.994762 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:36.997222 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:37.001770 (+ 4548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:37.001779 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:37.001793 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:37.002305 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:37.002899 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:37.002904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:37.002906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:37.003124 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:37.004211 (+ 1087us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:37.004218 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:37.004527 (+ 309us) client_negotiation.cc:769] Sending connection context
0927 07:03:37.004766 (+ 239us) client_negotiation.cc:240] Negotiation successful
0927 07:03:37.005203 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":422,"threads_started":1}
I20260927 07:03:37.006567 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:36.982669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:54695 (local address 127.13.121.254:34703)
0927 07:03:36.983396 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:36.983412 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:36.983925 (+ 513us) server_negotiation.cc:407] Connection header received
0927 07:03:36.986647 (+ 2722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:36.986689 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:36.987100 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:36.987482 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:36.989089 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:36.990929 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:36.992624 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:36.993621 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:36.993817 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:36.997589 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:36.997602 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:36.997626 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:36.997699 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:37.001584 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:37.002475 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:37.002486 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:37.002493 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:37.002746 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:37.003284 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:37.003289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:37.003293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:37.004045 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:37.004265 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:37.005044 (+ 779us) server_negotiation.cc:299] Negotiation successful
0927 07:03:37.005471 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":264,"threads_started":1}
I20260927 07:03:37.009413 28366 heartbeater.cc:347] Connected to a master server at 127.13.121.254:34703
I20260927 07:03:37.009760 28366 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:37.010593 28366 heartbeater.cc:511] Master 127.13.121.254:34703 requested a full tablet report, sending...
I20260927 07:03:37.013108 28181 ts_manager.cc:194] Registered new tserver with Master: 64abfceb9020451192fd2e81881aee93 (127.13.121.193:37335)
I20260927 07:03:37.014642 28181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:54695
W20260927 07:03:37.310779 28370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:37.311081 28370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:37.311229 28370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:37.318616 28370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:37.318732 28370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:37.318877 28370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:03:37.328758 28370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:38137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=35007
--tserver_master_addrs=127.13.121.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:37.330070 28370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:37.331383 28370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:37.344165 28375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:37.344120 28378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:37.344208 28376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:37.348704 28370 server_base.cc:1034] running on GCE node
I20260927 07:03:37.349335 28370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:37.350322 28370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:37.351514 28370 hybrid_clock.cc:648] HybridClock initialized: now 1790492617351464 us; error 45 us; skew 500 ppm
I20260927 07:03:37.351836 28370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:37 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:37.354687 28370 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:03:37.356145 28370 webserver.cc:460] Webserver started at http://127.13.121.194:35007/ using document root <none> and password file <none>
I20260927 07:03:37.356842 28370 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:37.356953 28370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:37.362115 28370 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 07:03:37.364673 28385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:37.365931 28370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:03:37.366072 28370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "c9be399be7f1481a8131347c2ab2fa8b"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "75e9cf1cb8a79b83c8b1a681d32c80a6"
server_key_iv: "6c9c7dd8d6cf9ad50e7f7993fb028a15"
server_key_version: "encryptionkey@0"
I20260927 07:03:37.366982 28370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:37.383682 28370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:37.387684 28370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:37.387912 28370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:37.388831 28370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:37.390141 28370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:37.390226 28370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:37.390305 28370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:37.390357 28370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:37.420239 28370 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:38137
I20260927 07:03:37.420308 28497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:38137 every 8 connection(s)
I20260927 07:03:37.421627 28370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:03:37.425450 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 28370
I20260927 07:03:37.426071 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:33893
--local_ip_for_outbound_sockets=127.13.121.195
--tserver_master_addrs=127.13.121.254:34703
--webserver_port=40763
--webserver_interface=127.13.121.195
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:03:37 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:37.440737 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.424117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:46045 (local address 127.13.121.254:34703)
0927 07:03:37.424277 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:37.424281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:37.425364 (+ 1083us) server_negotiation.cc:407] Connection header received
0927 07:03:37.427262 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:37.427268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:37.427359 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:37.427450 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:37.428911 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.429679 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.431284 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.431702 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.431814 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.435270 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:37.435276 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:37.435279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:37.435314 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:37.437626 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:37.438471 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:37.438475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:37.438477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:37.438588 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:37.439016 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:37.439019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:37.439021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:37.439328 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:37.439439 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:37.440304 (+ 865us) server_negotiation.cc:299] Negotiation successful
0927 07:03:37.440483 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:03:37.442225 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.424504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:46045)
0927 07:03:37.425194 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:37.425232 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:37.426960 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:37.427701 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:37.427709 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:37.428206 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:37.428740 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.428757 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.429837 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.429840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.431105 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.431117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.431918 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.431926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.432152 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.432765 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:37.432786 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:37.435002 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:37.437773 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:37.437782 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:37.437800 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:37.438368 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:37.438681 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:37.438686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:37.438690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:37.438923 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:37.439439 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:37.439448 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:37.439883 (+ 435us) client_negotiation.cc:769] Sending connection context
0927 07:03:37.440210 (+ 327us) client_negotiation.cc:240] Negotiation successful
0927 07:03:37.440765 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":264,"threads_started":1}
I20260927 07:03:37.443921 28498 heartbeater.cc:347] Connected to a master server at 127.13.121.254:34703
I20260927 07:03:37.444267 28498 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:37.445039 28498 heartbeater.cc:511] Master 127.13.121.254:34703 requested a full tablet report, sending...
I20260927 07:03:37.446575 28181 ts_manager.cc:194] Registered new tserver with Master: c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137)
I20260927 07:03:37.447425 28181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:46045
W20260927 07:03:37.744133 28501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:37.744436 28501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:37.744581 28501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:37.752112 28501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:37.752246 28501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:37.752398 28501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:03:37.762408 28501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:33893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=40763
--tserver_master_addrs=127.13.121.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:37.763756 28501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:37.765067 28501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:37.777994 28506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:37.778043 28509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:37.778062 28507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:37.779039 28501 server_base.cc:1034] running on GCE node
I20260927 07:03:37.779634 28501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:37.780579 28501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:37.781770 28501 hybrid_clock.cc:648] HybridClock initialized: now 1790492617781719 us; error 40 us; skew 500 ppm
I20260927 07:03:37.782051 28501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:37 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:37.785110 28501 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:03:37.786494 28501 webserver.cc:460] Webserver started at http://127.13.121.195:40763/ using document root <none> and password file <none>
I20260927 07:03:37.787182 28501 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:37.787313 28501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:37.792768 28501 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 07:03:37.795315 28516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:37.796599 28501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:03:37.796737 28501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "79c1aac6ee35417f82cc6bada1eae949"
format_stamp: "Formatted at 2026-09-27 07:03:35 on dist-test-slave-p6hz"
server_key: "c03655822e0b45c19b3330b978b7bdfe"
server_key_iv: "9b548864d763a8ef893cfc83d4b5597a"
server_key_version: "encryptionkey@0"
I20260927 07:03:37.797719 28501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:37.812345 28501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:37.816545 28501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:37.816817 28501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:37.817845 28501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:37.819268 28501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:37.819365 28501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:37.819458 28501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:37.819528 28501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:37.848800 28501 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:33893
I20260927 07:03:37.848871 28628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:33893 every 8 connection(s)
I20260927 07:03:37.850265 28501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:03:37.857796 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 28501
Sep 27 07:03:37 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:37.872287 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.852715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:52177 (local address 127.13.121.254:34703)
0927 07:03:37.852890 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:37.852895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:37.853973 (+ 1078us) server_negotiation.cc:407] Connection header received
0927 07:03:37.855868 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:37.855874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:37.855957 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:37.856062 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:37.857722 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.858713 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.860829 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.861290 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.861427 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.865464 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:37.865472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:37.865477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:37.865520 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:37.868554 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:37.869573 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:37.869578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:37.869582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:37.869742 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:37.870298 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:37.870301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:37.870304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:37.870605 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:37.870714 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:37.871709 (+ 995us) server_negotiation.cc:299] Negotiation successful
0927 07:03:37.871932 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":61}
I20260927 07:03:37.873418 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.853177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:52177)
0927 07:03:37.853830 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:37.853867 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:37.855607 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:37.856319 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:37.856331 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:37.856851 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:37.857532 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.857556 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.858914 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.858920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.860699 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.860712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.861507 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.861515 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.861799 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.862465 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:37.862495 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:37.865226 (+ 2731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:37.868736 (+ 3510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:37.868744 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:37.868760 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:37.869425 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:37.869861 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:37.869866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:37.869869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:37.870171 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:37.870737 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:37.870746 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:37.871271 (+ 525us) client_negotiation.cc:769] Sending connection context
0927 07:03:37.871555 (+ 284us) client_negotiation.cc:240] Negotiation successful
0927 07:03:37.871983 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":247,"threads_started":1}
I20260927 07:03:37.874155 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.858758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.13.121.254:34703)
0927 07:03:37.859439 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:37.859446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:37.859584 (+ 138us) server_negotiation.cc:407] Connection header received
0927 07:03:37.859729 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:37.859734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:37.859806 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:37.859910 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:37.861389 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.862300 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.864167 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.864644 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.864748 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.868020 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:37.868027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:37.868030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:37.868074 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:37.871191 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:37.872052 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:37.872057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:37.872060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:37.872218 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:37.872636 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:37.872642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:37.872646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:37.872990 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:37.873249 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:37.873525 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 07:03:37.873742 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":361,"thread_start_us":245,"threads_started":1}
I20260927 07:03:37.875003 28629 heartbeater.cc:347] Connected to a master server at 127.13.121.254:34703
I20260927 07:03:37.875355 28629 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:37.876116 28629 heartbeater.cc:511] Master 127.13.121.254:34703 requested a full tablet report, sending...
I20260927 07:03:37.877521 28181 ts_manager.cc:194] Registered new tserver with Master: 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893)
I20260927 07:03:37.878300 28181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:52177
I20260927 07:03:37.888772 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:03:37.895503 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.889708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.13.121.254:34703)
0927 07:03:37.889925 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:37.889932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:37.890242 (+ 310us) server_negotiation.cc:407] Connection header received
0927 07:03:37.890452 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:37.890456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:37.890600 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:37.890733 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:03:37.891178 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.891859 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.893296 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.893726 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.893842 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.894319 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:03:37.894845 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:03:37.894952 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:37.895115 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 07:03:37.895232 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:03:37.896550 28181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:03:37.904467 28181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:03:37.927340 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.917487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.13.121.194:38137)
0927 07:03:37.919095 (+ 1608us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:37.919104 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:37.919159 (+ 55us) server_negotiation.cc:407] Connection header received
0927 07:03:37.919272 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:37.919280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:37.919568 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:37.920523 (+ 955us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:37.920801 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.922378 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.924999 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.926209 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.926391 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.926584 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:37.926808 (+ 224us) server_negotiation.cc:299] Negotiation successful
0927 07:03:37.926975 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1429,"thread_start_us":1297,"threads_started":1}
I20260927 07:03:37.927419 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.916923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:38137 (local address 127.0.0.1:41292)
0927 07:03:37.918460 (+ 1537us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:37.918478 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:37.918589 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:37.919830 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:37.919840 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:37.919871 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:37.920294 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.920307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.922544 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.922549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.924809 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.924826 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.926395 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.926401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.926584 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.926639 (+ 55us) client_negotiation.cc:769] Sending connection context
0927 07:03:37.926750 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 07:03:37.926949 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1422,"mutex_wait_us":112,"thread_start_us":1284,"threads_started":1}
I20260927 07:03:37.928043 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.919178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.0.0.1:46434)
0927 07:03:37.919645 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:37.919661 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:37.919773 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:37.920652 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:37.920656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:37.920679 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:37.921056 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.921069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.922637 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.922641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.925338 (+ 2697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.925349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.927205 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.927213 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.927418 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.927451 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 07:03:37.927520 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 07:03:37.927655 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":30,"thread_start_us":637,"threads_started":1}
I20260927 07:03:37.928355 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.916750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33893 (local address 127.0.0.1:43256)
0927 07:03:37.917571 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:37.917622 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:37.917798 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:37.920192 (+ 2394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:37.920197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:37.920224 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:37.920564 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.920575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.922565 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.922570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.925276 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:37.925296 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.927476 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.927482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:37.927636 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.927664 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 07:03:37.927754 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 07:03:37.927879 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":244,"thread_start_us":410,"threads_started":1}
I20260927 07:03:37.928614 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.916957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.13.121.195:33893)
0927 07:03:37.919401 (+ 2444us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:37.919408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:37.919439 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:03:37.919526 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:37.919533 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:37.919753 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:37.919935 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:37.920767 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.922382 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.925481 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.926918 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.927109 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.927310 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:37.928141 (+ 831us) server_negotiation.cc:299] Negotiation successful
0927 07:03:37.928317 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2289,"thread_start_us":2119,"threads_started":1}
I20260927 07:03:37.929131 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:37.919651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46434 (local address 127.13.121.193:37335)
0927 07:03:37.920224 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:37.920230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:37.920252 (+ 22us) server_negotiation.cc:407] Connection header received
0927 07:03:37.920321 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:37.920326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:37.920487 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:37.920639 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:37.921262 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.922507 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.925502 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:37.926918 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:37.928457 (+ 1539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:37.928624 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:37.928694 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 07:03:37.928863 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":314,"threads_started":1}
I20260927 07:03:37.931636 28433 tablet_service.cc:1467] Processing CreateTablet for tablet 3aaef909438c4ea489fd40c0ba27d6e9 (DEFAULT_TABLE table=test-table [id=00af4b9ea0104f289b1ffd2cb5e1e0ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:37.932792 28433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3aaef909438c4ea489fd40c0ba27d6e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:37.932493 28564 tablet_service.cc:1467] Processing CreateTablet for tablet 3aaef909438c4ea489fd40c0ba27d6e9 (DEFAULT_TABLE table=test-table [id=00af4b9ea0104f289b1ffd2cb5e1e0ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:37.932521 28301 tablet_service.cc:1467] Processing CreateTablet for tablet 3aaef909438c4ea489fd40c0ba27d6e9 (DEFAULT_TABLE table=test-table [id=00af4b9ea0104f289b1ffd2cb5e1e0ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:37.933800 28564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3aaef909438c4ea489fd40c0ba27d6e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:37.933801 28301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3aaef909438c4ea489fd40c0ba27d6e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:37.944450 28645 tablet_bootstrap.cc:492] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: Bootstrap starting.
I20260927 07:03:37.947999 28645 tablet_bootstrap.cc:654] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:37.949020 28645 log.cc:824] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:37.951856 28645 tablet_bootstrap.cc:492] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: No bootstrap required, opened a new log
I20260927 07:03:37.952106 28645 ts_tablet_manager.cc:1397] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260927 07:03:37.959270 28647 tablet_bootstrap.cc:492] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: Bootstrap starting.
I20260927 07:03:37.960232 28646 tablet_bootstrap.cc:492] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Bootstrap starting.
I20260927 07:03:37.961135 28645 raft_consensus.cc:348] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:37.961508 28645 raft_consensus.cc:374] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:37.961618 28645 raft_consensus.cc:729] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9be399be7f1481a8131347c2ab2fa8b, State: Initialized, Role: FOLLOWER
I20260927 07:03:37.962306 28645 consensus_queue.cc:260] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:37.963102 28647 tablet_bootstrap.cc:654] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:37.963727 28646 tablet_bootstrap.cc:654] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:37.964525 28647 log.cc:824] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:37.964963 28646 log.cc:824] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:37.966064 28498 heartbeater.cc:503] Master 127.13.121.254:34703 was elected leader, sending a full tablet report...
I20260927 07:03:37.966626 28645 ts_tablet_manager.cc:1428] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260927 07:03:37.968765 28647 tablet_bootstrap.cc:492] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: No bootstrap required, opened a new log
I20260927 07:03:37.969069 28647 ts_tablet_manager.cc:1397] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260927 07:03:37.970162 28646 tablet_bootstrap.cc:492] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: No bootstrap required, opened a new log
I20260927 07:03:37.970521 28646 ts_tablet_manager.cc:1397] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260927 07:03:37.979777 28647 raft_consensus.cc:348] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:37.980279 28647 raft_consensus.cc:374] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:37.980382 28647 raft_consensus.cc:729] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c1aac6ee35417f82cc6bada1eae949, State: Initialized, Role: FOLLOWER
I20260927 07:03:37.981063 28647 consensus_queue.cc:260] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:37.981389 28646 raft_consensus.cc:348] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:37.981886 28646 raft_consensus.cc:374] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:37.982031 28646 raft_consensus.cc:729] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64abfceb9020451192fd2e81881aee93, State: Initialized, Role: FOLLOWER
I20260927 07:03:37.982337 28629 heartbeater.cc:503] Master 127.13.121.254:34703 was elected leader, sending a full tablet report...
I20260927 07:03:37.982960 28647 ts_tablet_manager.cc:1428] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: Time spent starting tablet: real 0.014s user 0.005s sys 0.009s
I20260927 07:03:37.982738 28646 consensus_queue.cc:260] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:37.984109 28366 heartbeater.cc:503] Master 127.13.121.254:34703 was elected leader, sending a full tablet report...
I20260927 07:03:37.984735 28646 ts_tablet_manager.cc:1428] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Time spent starting tablet: real 0.014s user 0.004s sys 0.009s
I20260927 07:03:38.053329 28649 raft_consensus.cc:482] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:38.053689 28649 raft_consensus.cc:504] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:38.055450 28649 leader_election.cc:288] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893), 64abfceb9020451192fd2e81881aee93 (127.13.121.193:37335)
I20260927 07:03:38.062026 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.056105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:37721 (local address 127.13.121.193:37335)
0927 07:03:38.056290 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:38.056295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:38.056836 (+ 541us) server_negotiation.cc:407] Connection header received
0927 07:03:38.057025 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:38.057028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:38.057089 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:38.057159 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:38.057783 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.058538 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.060290 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.061076 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.061210 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:38.061286 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:38.061646 (+ 360us) server_negotiation.cc:299] Negotiation successful
0927 07:03:38.061767 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:03:38.062019 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.055809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:52767 (local address 127.13.121.195:33893)
0927 07:03:38.056012 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:38.056018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:38.056742 (+ 724us) server_negotiation.cc:407] Connection header received
0927 07:03:38.056946 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:38.056952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:38.057011 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:38.057083 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:38.057654 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.058432 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.060299 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.061085 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.061233 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:38.061306 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:38.061657 (+ 351us) server_negotiation.cc:299] Negotiation successful
0927 07:03:38.061780 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:03:38.062158 28654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.055626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33893 (local address 127.13.121.194:52767)
0927 07:03:38.056610 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:38.056631 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:38.056811 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:38.057116 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:38.057120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:38.057143 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:38.057515 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:38.057527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.058561 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.058567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:38.060153 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:38.060165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.061328 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.061336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:38.061516 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:38.061539 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 07:03:38.061615 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 07:03:38.061747 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":47,"spinlock_wait_cycles":1920,"thread_start_us":586,"threads_started":1}
I20260927 07:03:38.062438 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.055877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.13.121.194:37721)
0927 07:03:38.056735 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:38.056748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:38.056891 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:38.057186 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:38.057189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:38.057244 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:38.057608 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:38.057618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.058668 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.058673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:38.060139 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:38.060151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.061310 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.061316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:38.061476 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:38.061517 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 07:03:38.061600 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 07:03:38.061723 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":22,"spinlock_wait_cycles":33280,"thread_start_us":547,"threads_started":1}
I20260927 07:03:38.063345 28584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aaef909438c4ea489fd40c0ba27d6e9" candidate_uuid: "c9be399be7f1481a8131347c2ab2fa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c1aac6ee35417f82cc6bada1eae949" is_pre_election: true
I20260927 07:03:38.063462 28321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aaef909438c4ea489fd40c0ba27d6e9" candidate_uuid: "c9be399be7f1481a8131347c2ab2fa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64abfceb9020451192fd2e81881aee93" is_pre_election: true
I20260927 07:03:38.063989 28584 raft_consensus.cc:2454] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9be399be7f1481a8131347c2ab2fa8b in term 0.
I20260927 07:03:38.063992 28321 raft_consensus.cc:2454] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9be399be7f1481a8131347c2ab2fa8b in term 0.
I20260927 07:03:38.064697 28386 leader_election.cc:302] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64abfceb9020451192fd2e81881aee93, c9be399be7f1481a8131347c2ab2fa8b; no voters:
I20260927 07:03:38.065090 28649 raft_consensus.cc:2781] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:03:38.065218 28649 raft_consensus.cc:482] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:03:38.065310 28649 raft_consensus.cc:3037] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:38.067564 28649 raft_consensus.cc:504] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:38.068192 28649 leader_election.cc:288] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [CANDIDATE]: Term 1 election: Requested vote from peers 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893), 64abfceb9020451192fd2e81881aee93 (127.13.121.193:37335)
I20260927 07:03:38.068794 28321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aaef909438c4ea489fd40c0ba27d6e9" candidate_uuid: "c9be399be7f1481a8131347c2ab2fa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64abfceb9020451192fd2e81881aee93"
I20260927 07:03:38.068787 28584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aaef909438c4ea489fd40c0ba27d6e9" candidate_uuid: "c9be399be7f1481a8131347c2ab2fa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c1aac6ee35417f82cc6bada1eae949"
I20260927 07:03:38.069022 28584 raft_consensus.cc:3037] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:38.069022 28321 raft_consensus.cc:3037] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:38.072271 28584 raft_consensus.cc:2454] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9be399be7f1481a8131347c2ab2fa8b in term 1.
I20260927 07:03:38.072271 28321 raft_consensus.cc:2454] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9be399be7f1481a8131347c2ab2fa8b in term 1.
I20260927 07:03:38.072773 28389 leader_election.cc:302] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 64abfceb9020451192fd2e81881aee93, 79c1aac6ee35417f82cc6bada1eae949, c9be399be7f1481a8131347c2ab2fa8b; no voters:
I20260927 07:03:38.073148 28649 raft_consensus.cc:2781] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:03:38.073575 28649 raft_consensus.cc:686] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 1 LEADER]: Becoming Leader. State: Replica: c9be399be7f1481a8131347c2ab2fa8b, State: Running, Role: LEADER
I20260927 07:03:38.074046 28649 consensus_queue.cc:237] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:38.078941 28181 catalog_manager.cc:5495] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b reported cstate change: term changed from 0 to 1, leader changed from <none> to c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194). New cstate: current_term: 1 leader_uuid: "c9be399be7f1481a8131347c2ab2fa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } health_report { overall_health: UNKNOWN } } }
W20260927 07:03:38.101866 28630 tablet.cc:2369] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:03:38.128093 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.122610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41300 (local address 127.13.121.194:38137)
0927 07:03:38.122832 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:38.122839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:38.122857 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:03:38.122967 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:38.122973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:38.123110 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:38.123294 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:03:38.123715 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.124444 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.125914 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.126336 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.126447 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:38.126913 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:03:38.127510 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:03:38.127641 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:38.127740 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 07:03:38.127825 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:03:38.136651 28584 raft_consensus.cc:1261] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [term 1 FOLLOWER]: Refusing update from remote peer c9be399be7f1481a8131347c2ab2fa8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:38.136651 28321 raft_consensus.cc:1261] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Refusing update from remote peer c9be399be7f1481a8131347c2ab2fa8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:38.137858 28656 consensus_queue.cc:1035] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:03:38.138240 28649 consensus_queue.cc:1035] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:03:38.149310 28659 mvcc.cc:205] Tried to move back new op lower bound from 7333857763877019648 to 7333857763635838976. Current Snapshot: MvccSnapshot[applied={T|T < 7333857763877019648}]
I20260927 07:03:38.166030 28179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:03:38.166333 28179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:03:38.170288 28179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Sending DeleteTablet for 3 replicas of tablet 3aaef909438c4ea489fd40c0ba27d6e9
I20260927 07:03:38.171561 28564 tablet_service.cc:1514] Processing DeleteTablet for tablet 3aaef909438c4ea489fd40c0ba27d6e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:38 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:43256
I20260927 07:03:38.171787 28301 tablet_service.cc:1514] Processing DeleteTablet for tablet 3aaef909438c4ea489fd40c0ba27d6e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:38 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:46434
I20260927 07:03:38.172037 28433 tablet_service.cc:1514] Processing DeleteTablet for tablet 3aaef909438c4ea489fd40c0ba27d6e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:38 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:41292
I20260927 07:03:38.172933 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 28218
I20260927 07:03:38.182020 28668 tablet_replica.cc:330] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: stopping tablet replica
I20260927 07:03:38.182452 28668 raft_consensus.cc:2229] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:03:38.183029 28668 raft_consensus.cc:2258] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:38.184635 28670 tablet_replica.cc:330] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: stopping tablet replica
I20260927 07:03:38.185248 28670 raft_consensus.cc:2229] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:03:38.186636 28670 raft_consensus.cc:2258] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:38.188305 28668 ts_tablet_manager.cc:1905] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:38.193897 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.174904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46438 (local address 127.13.121.193:37335)
0927 07:03:38.175093 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:38.175099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:38.175117 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:03:38.175203 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:38.175209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:38.175298 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:38.175419 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:38.177336 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.178671 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.180791 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.181544 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.181677 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:38.185314 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:38.185334 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:38.185353 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:38.185403 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:38.189633 (+ 4230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:38.190580 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:38.190590 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:38.190593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:38.190760 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:38.191249 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:38.191255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:38.191258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:38.191769 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:38.192028 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:38.193276 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0927 07:03:38.193521 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 07:03:38.194919 28361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:38.197256 28670 ts_tablet_manager.cc:1905] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:38.205869 28670 ts_tablet_manager.cc:1918] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:38.206526 28668 ts_tablet_manager.cc:1918] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:38.206712 28668 log.cc:1196] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/3aaef909438c4ea489fd40c0ba27d6e9
I20260927 07:03:38.207444 28668 ts_tablet_manager.cc:1939] T 3aaef909438c4ea489fd40c0ba27d6e9 P 79c1aac6ee35417f82cc6bada1eae949: Deleting consensus metadata
I20260927 07:03:38.209482 28670 log.cc:1196] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/3aaef909438c4ea489fd40c0ba27d6e9
I20260927 07:03:38.209692 28168 catalog_manager.cc:4841] TS 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893): tablet 3aaef909438c4ea489fd40c0ba27d6e9 (table test-table [id=00af4b9ea0104f289b1ffd2cb5e1e0ba]) successfully deleted
I20260927 07:03:38.210193 28670 ts_tablet_manager.cc:1939] T 3aaef909438c4ea489fd40c0ba27d6e9 P c9be399be7f1481a8131347c2ab2fa8b: Deleting consensus metadata
I20260927 07:03:38.211642 28165 catalog_manager.cc:4841] TS c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137): tablet 3aaef909438c4ea489fd40c0ba27d6e9 (table test-table [id=00af4b9ea0104f289b1ffd2cb5e1e0ba]) successfully deleted
I20260927 07:03:38.322654 28669 tablet_replica.cc:330] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: stopping tablet replica
I20260927 07:03:38.322975 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 28218
W20260927 07:03:38.337786 28165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.121.193:37335 (error 108)
I20260927 07:03:38.338073 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 28370
I20260927 07:03:38.339970 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.339324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.0.0.1:46454)
0927 07:03:38.339539 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:38.339663 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260927 07:03:38.340420 28165 catalog_manager.cc:4584] TS 64abfceb9020451192fd2e81881aee93 (127.13.121.193:37335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3aaef909438c4ea489fd40c0ba27d6e9: Network error: Client connection negotiation failed: client connection to 127.13.121.193:37335: connect: Connection refused (error 111)
I20260927 07:03:38.352830 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.338900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.13.121.194:38137)
0927 07:03:38.339092 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:38.339096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:38.339195 (+ 99us) server_negotiation.cc:407] Connection header received
0927 07:03:38.339419 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:38.339425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:38.339527 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:38.339628 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:38.341276 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.342100 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.343570 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.344051 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.344165 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:38.346728 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:38.346743 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:38.346759 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:38.346795 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:38.349851 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:38.350623 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:38.350630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:38.350632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:38.350785 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:38.351255 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:38.351262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:38.351264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:38.351640 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:38.351960 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:38.352257 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 07:03:38.352483 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 07:03:38.353554 28493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:38.394279 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.393758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.0.0.1:46468)
0927 07:03:38.393997 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:38.394104 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260927 07:03:38.395386 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.394952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.0.0.1:46476)
0927 07:03:38.395131 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:38.395264 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260927 07:03:38.456102 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.455605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.0.0.1:46480)
0927 07:03:38.455807 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:38.455914 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260927 07:03:38.457149 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.456816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.0.0.1:46490)
0927 07:03:38.456949 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:38.457026 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 07:03:38.467211 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 28370
I20260927 07:03:38.483058 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 28501
I20260927 07:03:38.496609 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.483859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.13.121.195:33893)
0927 07:03:38.484029 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:38.484035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:38.484074 (+ 39us) server_negotiation.cc:407] Connection header received
0927 07:03:38.484270 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:38.484277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:38.484356 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:38.484473 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:38.485766 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.486487 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.488065 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:38.488465 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:38.488577 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:38.491117 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:38.491152 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:38.491174 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:38.491218 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:38.493853 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:38.494559 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:38.494567 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:38.494569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:38.494706 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:38.495065 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:38.495070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:38.495073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:38.495495 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:38.495786 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:38.496063 (+ 277us) server_negotiation.cc:299] Negotiation successful
0927 07:03:38.496288 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 07:03:38.497262 28624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:38.538689 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.538036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.0.0.1:46494)
0927 07:03:38.538325 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:38.538460 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260927 07:03:38.539978 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:38.539519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:37335 (local address 127.0.0.1:46498)
0927 07:03:38.539731 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:38.539825 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260927 07:03:38.587044 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 28501
I20260927 07:03:38.601009 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 28148
I20260927 07:03:38.602236 28211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:38.716697 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 28148
I20260927 07:03:38.735080 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:34703
--webserver_interface=127.13.121.254
--webserver_port=41773
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:34703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:39.058125 28676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:39.058454 28676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:39.058597 28676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:39.066037 28676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:39.066170 28676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:39.066250 28676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:39.066323 28676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:39.066381 28676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:39.076494 28676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:34703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:34703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=41773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:39.077884 28676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:39.079241 28676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:39.091293 28684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:39.091372 28682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:39.091348 28681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:39.091465 28676 server_base.cc:1034] running on GCE node
I20260927 07:03:39.092847 28676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:39.094316 28676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:39.095563 28676 hybrid_clock.cc:648] HybridClock initialized: now 1790492619095483 us; error 43 us; skew 500 ppm
I20260927 07:03:39.095920 28676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:39 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492619, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:39.105242 28676 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:03:39.107000 28676 webserver.cc:460] Webserver started at http://127.13.121.254:41773/ using document root <none> and password file <none>
I20260927 07:03:39.107788 28676 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:39.107918 28676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:39.113792 28676 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 07:03:39.117059 28691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:39.118589 28676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:03:39.118793 28676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "d6f9251dce064af48933aacb03ff9bbe"
format_stamp: "Formatted at 2026-09-27 07:03:33 on dist-test-slave-p6hz"
server_key: "617268d123fef3114df9ae6ebbc05a66"
server_key_iv: "81d79c74b71bc2a1e6d1e546a5b16d94"
server_key_version: "encryptionkey@0"
I20260927 07:03:39.119884 28676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:39.135079 28676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:39.139382 28676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:39.139664 28676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:39.157423 28676 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:34703
I20260927 07:03:39.157457 28742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:34703 every 8 connection(s)
I20260927 07:03:39.158910 28676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:03:39.165774 28743 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:03:39.166595 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 28676
I20260927 07:03:39.167395 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:37335
--local_ip_for_outbound_sockets=127.13.121.193
--tserver_master_addrs=127.13.121.254:34703
--webserver_port=36437
--webserver_interface=127.13.121.193
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:03:39.169237 28743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Bootstrap starting.
I20260927 07:03:39.189563 28743 log.cc:824] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:39.205917 28743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:03:39.206452 28743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe: Bootstrap complete.
I20260927 07:03:39.215639 28743 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:39.216598 28743 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6f9251dce064af48933aacb03ff9bbe, State: Initialized, Role: FOLLOWER
I20260927 07:03:39.217250 28743 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:39.217504 28743 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:03:39.217649 28743 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:03:39.217844 28743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 2 FOLLOWER]: Advancing to term 3
I20260927 07:03:39.222805 28743 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:39.223368 28743 leader_election.cc:302] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6f9251dce064af48933aacb03ff9bbe; no voters:
I20260927 07:03:39.224092 28743 leader_election.cc:288] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 07:03:39.224233 28747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 3 FOLLOWER]: Leader election won for term 3
I20260927 07:03:39.226325 28743 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:39.227478 28747 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [term 3 LEADER]: Becoming Leader. State: Replica: d6f9251dce064af48933aacb03ff9bbe, State: Running, Role: LEADER
I20260927 07:03:39.228221 28747 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } }
I20260927 07:03:39.231002 28749 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6f9251dce064af48933aacb03ff9bbe. Latest consensus state: current_term: 3 leader_uuid: "d6f9251dce064af48933aacb03ff9bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } } }
I20260927 07:03:39.231099 28748 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d6f9251dce064af48933aacb03ff9bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6f9251dce064af48933aacb03ff9bbe" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34703 } } }
I20260927 07:03:39.231457 28748 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:39.231521 28749 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6f9251dce064af48933aacb03ff9bbe [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:39.247810 28757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:03:39.257141 28757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:03:39.258687 28757 catalog_manager.cc:1259] Loaded cluster ID: 86936951dda545b78df036ebb65eeb84
I20260927 07:03:39.258867 28757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260927 07:03:39.261078 28757 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
I20260927 07:03:39.263887 28764 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f60223f99ad google::LogMessage::Fail() at ??:0
@ 0x7f60223fc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f60223f94de google::LogMessage::Flush() at ??:0
@ 0x7f60223faa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6037e2c512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6037e29976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6037eb1610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f6037eb1439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6025045345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f602504891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6025048799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6025023eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f60282fd6db start_thread at ??:0
@ 0x7f602129361f clone at ??:0
W20260927 07:03:39.548870 28745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:39.549160 28745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:39.549270 28745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:39.556720 28745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:39.556849 28745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:39.556967 28745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:03:39.567009 28745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:37335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=36437
--tserver_master_addrs=127.13.121.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:39.568373 28745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:39.569689 28745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:39.582554 28784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:39.582513 28785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:39.582583 28787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:39.583365 28745 server_base.cc:1034] running on GCE node
I20260927 07:03:39.584110 28745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:39.585048 28745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:39.586283 28745 hybrid_clock.cc:648] HybridClock initialized: now 1790492619586216 us; error 50 us; skew 500 ppm
I20260927 07:03:39.586655 28745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:39 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492619, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:39.589993 28745 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:03:39.591599 28745 webserver.cc:460] Webserver started at http://127.13.121.193:36437/ using document root <none> and password file <none>
I20260927 07:03:39.592299 28745 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:39.592423 28745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:39.597942 28745 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260927 07:03:39.600899 28794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:39.602334 28745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:03:39.602512 28745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "64abfceb9020451192fd2e81881aee93"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "5a59f31a75a9fed3438cea680d18ce3f"
server_key_iv: "6bef08e7e1f330ccc79b4f62fa9f4bb4"
server_key_version: "encryptionkey@0"
I20260927 07:03:39.603585 28745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:39.632357 28745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:39.637040 28745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:39.637339 28745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:39.638433 28745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:39.640401 28801 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260927 07:03:39.646272 28745 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 07:03:39.646416 28745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260927 07:03:39.646615 28745 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 07:03:39.649963 28745 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 07:03:39.650051 28745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260927 07:03:39.650396 28801 tablet_bootstrap.cc:492] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Bootstrap starting.
I20260927 07:03:39.676331 28801 log.cc:824] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:39.688249 28745 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:37335
I20260927 07:03:39.688268 28908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:37335 every 8 connection(s)
I20260927 07:03:39.689543 28745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:03:39.689812 28801 tablet_bootstrap.cc:492] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:03:39.690474 28801 tablet_bootstrap.cc:492] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Bootstrap complete.
I20260927 07:03:39.690541 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 28745
I20260927 07:03:39.691397 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:38137
--local_ip_for_outbound_sockets=127.13.121.194
--tserver_master_addrs=127.13.121.254:34703
--webserver_port=35007
--webserver_interface=127.13.121.194
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:03:39.692082 28801 ts_tablet_manager.cc:1397] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Time spent bootstrapping tablet: real 0.042s user 0.036s sys 0.004s
I20260927 07:03:39.699780 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:39.692656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:54569)
0927 07:03:39.698086 (+ 5430us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:39.698446 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5103,"thread_start_us":4862,"threads_started":1}
W20260927 07:03:39.700716 28909 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:34703 (0 consecutive failures): Network error: Failed to ping master at 127.13.121.254:34703: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
I20260927 07:03:39.701819 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:39.701407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:41587)
0927 07:03:39.701605 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:39.701682 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260927 07:03:39.702782 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:39.702407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:53119)
0927 07:03:39.702560 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:39.702645 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 07:03:39.703084 28909 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:03:39.702768 28801 raft_consensus.cc:348] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:39.704049 28801 raft_consensus.cc:729] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64abfceb9020451192fd2e81881aee93, State: Initialized, Role: FOLLOWER
I20260927 07:03:39.704730 28801 consensus_queue.cc:260] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:39.706288 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:39.705954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:43107)
0927 07:03:39.706104 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:39.706169 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 07:03:39.706395 28801 ts_tablet_manager.cc:1428] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260927 07:03:39.707428 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:39.707002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:58251)
0927 07:03:39.707203 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:39.707282 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260927 07:03:40.020231 28912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:40.020540 28912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:40.020656 28912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:40.027977 28912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:40.028106 28912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:40.028251 28912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:03:40.038376 28912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:38137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=35007
--tserver_master_addrs=127.13.121.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:40.039788 28912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:40.041105 28912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:40.054961 28918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:40.054962 28921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:40.054904 28919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:40.055660 28912 server_base.cc:1034] running on GCE node
I20260927 07:03:40.056348 28912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:40.057461 28912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:40.058676 28912 hybrid_clock.cc:648] HybridClock initialized: now 1790492620058620 us; error 46 us; skew 500 ppm
I20260927 07:03:40.059006 28912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:40 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492620, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:40.062390 28912 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:03:40.063910 28912 webserver.cc:460] Webserver started at http://127.13.121.194:35007/ using document root <none> and password file <none>
I20260927 07:03:40.064589 28912 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:40.064692 28912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:40.070379 28912 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260927 07:03:40.075313 28928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:40.076726 28912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260927 07:03:40.076890 28912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "c9be399be7f1481a8131347c2ab2fa8b"
format_stamp: "Formatted at 2026-09-27 07:03:34 on dist-test-slave-p6hz"
server_key: "75e9cf1cb8a79b83c8b1a681d32c80a6"
server_key_iv: "6c9c7dd8d6cf9ad50e7f7993fb028a15"
server_key_version: "encryptionkey@0"
I20260927 07:03:40.077880 28912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:40.090397 28912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:40.094655 28912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:40.094914 28912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:40.096127 28912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:40.098982 28912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:40.099104 28912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:40.099433 28912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:40.099510 28912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:40.138134 28912 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:38137
I20260927 07:03:40.138190 29040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:38137 every 8 connection(s)
I20260927 07:03:40.139725 28912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:03:40.144574 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:40.142722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:56673)
0927 07:03:40.143494 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:40.143726 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":272,"threads_started":1}
W20260927 07:03:40.145231 29041 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:34703 (0 consecutive failures): Network error: Failed to ping master at 127.13.121.254:34703: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
I20260927 07:03:40.146044 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:40.145713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:44879)
0927 07:03:40.145858 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:40.145930 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 07:03:40.146879 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:40.146592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:54085)
0927 07:03:40.146704 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:40.146774 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 07:03:40.147106 29041 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:03:40.149397 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 28912
I20260927 07:03:40.149942 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:33893
--local_ip_for_outbound_sockets=127.13.121.195
--tserver_master_addrs=127.13.121.254:34703
--webserver_port=40763
--webserver_interface=127.13.121.195
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:40.469389 29044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:40.469700 29044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:40.469818 29044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:40.477190 29044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:40.477339 29044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:40.477499 29044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:03:40.487557 29044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:33893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=40763
--tserver_master_addrs=127.13.121.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:40.488840 29044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:40.490132 29044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:40.502986 29049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:40.502950 29050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:40.503015 29052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:40.503772 29044 server_base.cc:1034] running on GCE node
I20260927 07:03:40.504451 29044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:40.506006 29044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:40.507231 29044 hybrid_clock.cc:648] HybridClock initialized: now 1790492620507172 us; error 42 us; skew 500 ppm
I20260927 07:03:40.507566 29044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:40 dist-test-slave-p6hz krb5kdc[27596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492620, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:40.510748 29044 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:03:40.512271 29044 webserver.cc:460] Webserver started at http://127.13.121.195:40763/ using document root <none> and password file <none>
I20260927 07:03:40.512955 29044 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:40.513068 29044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:40.518559 29044 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260927 07:03:40.521713 29059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:40.523326 29044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:03:40.523521 29044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "79c1aac6ee35417f82cc6bada1eae949"
format_stamp: "Formatted at 2026-09-27 07:03:35 on dist-test-slave-p6hz"
server_key: "c03655822e0b45c19b3330b978b7bdfe"
server_key_iv: "9b548864d763a8ef893cfc83d4b5597a"
server_key_version: "encryptionkey@0"
I20260927 07:03:40.524534 29044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:40.537350 29044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:40.541723 29044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:40.541955 29044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:40.542909 29044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:40.544325 29044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:40.544430 29044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:40.544512 29044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:40.544564 29044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:40.575922 29044 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:33893
I20260927 07:03:40.575976 29171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:33893 every 8 connection(s)
I20260927 07:03:40.577399 29044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:03:40.581126 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 29044
I20260927 07:03:40.582546 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:40.580326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:34483)
0927 07:03:40.581152 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:40.581442 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":285,"threads_started":1}
W20260927 07:03:40.583299 29172 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:34703 (0 consecutive failures): Network error: Failed to ping master at 127.13.121.254:34703: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
I20260927 07:03:40.584105 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:40.583774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:42277)
0927 07:03:40.583921 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:40.583993 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 07:03:40.584941 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:40.584610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:45695)
0927 07:03:40.584751 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:40.584827 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260927 07:03:40.585166 29172 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:03:40.709103 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:40.708342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:53811)
0927 07:03:40.708805 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:40.708895 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":224,"threads_started":1}
I20260927 07:03:41.148532 29177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:41.147738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:54835)
0927 07:03:41.148192 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:41.148308 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":239,"threads_started":1}
I20260927 07:03:41.166574 29178 raft_consensus.cc:482] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:41.166896 29178 raft_consensus.cc:504] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:41.168589 29178 leader_election.cc:288] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893), c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137)
Sep 27 07:03:41 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492619, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
Sep 27 07:03:41 dist-test-slave-p6hz krb5kdc[27596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492619, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:03:41.197451 29179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:41.169030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:50095 (local address 127.13.121.195:33893)
0927 07:03:41.170444 (+ 1414us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:41.170460 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:41.170504 (+ 44us) server_negotiation.cc:407] Connection header received
0927 07:03:41.172494 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:41.172507 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:41.173039 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:41.173426 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:41.175564 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:41.177493 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:41.179662 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:41.180705 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:41.180911 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:41.190680 (+ 9769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:41.190691 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:41.190707 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:41.190748 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:41.193630 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:41.194582 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:41.194588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:41.194594 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:41.194757 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:41.195216 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:41.195220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:41.195222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:41.195756 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:41.195935 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:41.196719 (+ 784us) server_negotiation.cc:299] Negotiation successful
0927 07:03:41.197038 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1245,"thread_start_us":1129,"threads_started":1}
I20260927 07:03:41.197479 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:41.169424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:41331 (local address 127.13.121.194:38137)
0927 07:03:41.170817 (+ 1393us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:41.170830 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:41.171259 (+ 429us) server_negotiation.cc:407] Connection header received
0927 07:03:41.173401 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:41.173412 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:41.173863 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:41.174150 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:41.175593 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:41.177493 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:41.179707 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:41.180705 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:41.180911 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:41.190768 (+ 9857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:41.190777 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:41.190790 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:41.190833 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:41.193693 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:41.194571 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:41.194578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:41.194584 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:41.194767 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:41.195227 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:41.195231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:41.195233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:41.195759 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:41.195935 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:41.196714 (+ 779us) server_negotiation.cc:299] Negotiation successful
0927 07:03:41.197088 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1212,"thread_start_us":1051,"threads_started":1}
I20260927 07:03:41.197479 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:41.168780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33893 (local address 127.13.121.193:50095)
0927 07:03:41.169070 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:41.169121 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:41.169701 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:41.173576 (+ 3875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:41.173586 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:41.174255 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:41.175326 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:41.175355 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:41.177760 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:41.177766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:41.179485 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:41.179501 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:41.180960 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:41.180972 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:41.181330 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:41.186934 (+ 5604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:41.186987 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:41.190229 (+ 3242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:41.193799 (+ 3570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:41.193811 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:41.193827 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:41.194429 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:41.194870 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:41.194876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:41.194880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:41.195085 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:41.195902 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:41.195911 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:41.196217 (+ 306us) client_negotiation.cc:769] Sending connection context
0927 07:03:41.196452 (+ 235us) client_negotiation.cc:240] Negotiation successful
0927 07:03:41.197028 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":243,"spinlock_wait_cycles":4352}
I20260927 07:03:41.197935 29180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:41.169001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:38137 (local address 127.13.121.193:41331)
0927 07:03:41.171123 (+ 2122us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:41.171164 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:41.171277 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:41.174167 (+ 2890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:41.174173 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:41.174806 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:41.175326 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:41.175355 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:41.177761 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:41.177766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:41.179485 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:41.179501 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:41.181010 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:41.181018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:41.181326 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:41.186945 (+ 5619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:41.186987 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:41.190455 (+ 3468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:41.193850 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:41.193855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:41.193857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:41.194408 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:41.194880 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:41.194885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:41.194889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:41.195088 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:41.195901 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:41.195910 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:41.196217 (+ 307us) client_negotiation.cc:769] Sending connection context
0927 07:03:41.196452 (+ 235us) client_negotiation.cc:240] Negotiation successful
0927 07:03:41.197180 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1993,"mutex_wait_us":467,"spinlock_wait_cycles":50176,"thread_start_us":1790,"threads_started":1}
W20260927 07:03:41.200985 28795 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
W20260927 07:03:41.201735 28798 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
I20260927 07:03:41.201903 28798 leader_election.cc:302] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64abfceb9020451192fd2e81881aee93; no voters: 79c1aac6ee35417f82cc6bada1eae949, c9be399be7f1481a8131347c2ab2fa8b
I20260927 07:03:41.202287 29178 raft_consensus.cc:2726] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:03:41.586447 29182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:41.585627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:46109)
0927 07:03:41.586096 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:41.586233 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":228,"threads_started":1}
I20260927 07:03:41.710741 29183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:41.709987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:33335)
0927 07:03:41.710465 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:41.710562 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":246,"threads_started":1}
I20260927 07:03:42.150166 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:42.149483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:34621)
0927 07:03:42.149907 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:42.149993 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":219,"threads_started":1}
I20260927 07:03:42.588037 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:42.587324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:47189)
0927 07:03:42.587774 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:42.587860 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":226,"threads_started":1}
I20260927 07:03:42.712531 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:42.711692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:48849)
0927 07:03:42.712218 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:42.712343 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":225,"threads_started":1}
I20260927 07:03:43.151878 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:43.151087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:32963)
0927 07:03:43.151588 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:43.151701 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":280,"threads_started":1}
I20260927 07:03:43.314939 29188 raft_consensus.cc:482] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:43.315127 29188 raft_consensus.cc:504] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:43.315788 29188 leader_election.cc:288] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893), c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137)
W20260927 07:03:43.316754 28798 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
W20260927 07:03:43.316891 28795 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
I20260927 07:03:43.317040 28795 leader_election.cc:302] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64abfceb9020451192fd2e81881aee93; no voters: 79c1aac6ee35417f82cc6bada1eae949, c9be399be7f1481a8131347c2ab2fa8b
I20260927 07:03:43.317296 29188 raft_consensus.cc:2726] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:03:43.589686 29189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:43.588965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:57509)
0927 07:03:43.589423 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:43.589519 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":224,"threads_started":1}
I20260927 07:03:43.714325 29190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:43.713432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:42369)
0927 07:03:43.714008 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:43.714136 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":317,"threads_started":1}
I20260927 07:03:44.153594 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:44.152856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:48315)
0927 07:03:44.153312 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:44.153401 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":225,"threads_started":1}
I20260927 07:03:44.591607 29192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:44.590687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:52487)
0927 07:03:44.591251 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:44.591354 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":297,"threads_started":1}
I20260927 07:03:44.716341 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:44.715536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:48607)
0927 07:03:44.716057 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:44.716146 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":262,"threads_started":1}
I20260927 07:03:45.155378 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:45.154594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:38597)
0927 07:03:45.155067 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:45.155199 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":250,"threads_started":1}
I20260927 07:03:45.593277 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:45.592586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:39069)
0927 07:03:45.593012 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:45.593121 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":222,"threads_started":1}
I20260927 07:03:45.644083 29196 raft_consensus.cc:482] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:45.644268 29196 raft_consensus.cc:504] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:45.644879 29196 leader_election.cc:288] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893), c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137)
W20260927 07:03:45.645845 28798 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
W20260927 07:03:45.646044 28795 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
I20260927 07:03:45.646191 28795 leader_election.cc:302] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64abfceb9020451192fd2e81881aee93; no voters: 79c1aac6ee35417f82cc6bada1eae949, c9be399be7f1481a8131347c2ab2fa8b
I20260927 07:03:45.646443 29196 raft_consensus.cc:2726] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:03:45.718189 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:45.717258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:35537)
0927 07:03:45.717810 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:45.717922 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":267,"threads_started":1}
I20260927 07:03:46.157094 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:46.156330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:34677)
0927 07:03:46.156792 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:46.156896 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":234,"threads_started":1}
I20260927 07:03:46.594867 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:46.594125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:45689)
0927 07:03:46.594609 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:46.594692 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":237,"threads_started":1}
I20260927 07:03:46.719940 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:46.719120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:48535)
0927 07:03:46.719613 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:46.719729 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":225,"threads_started":1}
I20260927 07:03:47.159001 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:47.158153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:38793)
0927 07:03:47.158709 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:47.158810 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":232,"threads_started":1}
I20260927 07:03:47.586161 29202 raft_consensus.cc:482] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:47.586347 29202 raft_consensus.cc:504] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:47.586941 29202 leader_election.cc:288] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893), c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137)
W20260927 07:03:47.588037 28798 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
W20260927 07:03:47.588234 28795 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
I20260927 07:03:47.588346 28795 leader_election.cc:302] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64abfceb9020451192fd2e81881aee93; no voters: 79c1aac6ee35417f82cc6bada1eae949, c9be399be7f1481a8131347c2ab2fa8b
I20260927 07:03:47.588594 29202 raft_consensus.cc:2726] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:03:47.596583 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:47.595817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:51673)
0927 07:03:47.596314 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:47.596401 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":224,"threads_started":1}
I20260927 07:03:47.721752 29204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:47.720917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:35911)
0927 07:03:47.721407 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:47.721541 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":254,"threads_started":1}
I20260927 07:03:48.160745 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:48.160027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:50381)
0927 07:03:48.160456 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:48.160547 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":217,"threads_started":1}
I20260927 07:03:48.598162 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:48.597473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:54195)
0927 07:03:48.597900 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:48.597983 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":221,"threads_started":1}
I20260927 07:03:48.723588 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:48.722740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:55093)
0927 07:03:48.723277 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:48.723371 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":267,"threads_started":1}
I20260927 07:03:49.162474 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:49.161744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:54237)
0927 07:03:49.162236 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:49.162321 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":221,"threads_started":1}
I20260927 07:03:49.599921 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:49.599113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:46201)
0927 07:03:49.599665 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:49.599760 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":217,"threads_started":1}
I20260927 07:03:49.725371 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:49.724674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:37665)
0927 07:03:49.725121 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:49.725207 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":225,"threads_started":1}
I20260927 07:03:50.164206 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:50.163442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:45917)
0927 07:03:50.163929 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:50.164024 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":242,"threads_started":1}
I20260927 07:03:50.601636 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:50.600900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:51957)
0927 07:03:50.601363 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:50.601449 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":257,"threads_started":1}
I20260927 07:03:50.727258 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:50.726405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:58173)
0927 07:03:50.726914 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:50.727013 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":281,"threads_started":1}
I20260927 07:03:51.165977 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:51.165189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:41037)
0927 07:03:51.165716 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:51.165812 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":233,"threads_started":1}
I20260927 07:03:51.603263 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:51.602470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:56055)
0927 07:03:51.602949 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:51.603046 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":227,"threads_started":1}
I20260927 07:03:51.729101 29216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:51.728292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:55047)
0927 07:03:51.728767 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:51.728859 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":233,"threads_started":1}
I20260927 07:03:52.167788 29217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:52.166982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:57487)
0927 07:03:52.167481 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:52.167578 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":265,"threads_started":1}
I20260927 07:03:52.605057 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:52.604303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:56189)
0927 07:03:52.604777 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:52.604869 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":232,"threads_started":1}
I20260927 07:03:52.730847 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:52.730052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:38295)
0927 07:03:52.730579 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:52.730668 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":260,"threads_started":1}
I20260927 07:03:53.169485 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:53.168741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:52365)
0927 07:03:53.169229 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:53.169322 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":242,"threads_started":1}
I20260927 07:03:53.606755 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:53.605985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:51171)
0927 07:03:53.606447 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:53.606569 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":225,"threads_started":1}
I20260927 07:03:53.611479 29222 raft_consensus.cc:482] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:53.611673 29222 raft_consensus.cc:504] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c1aac6ee35417f82cc6bada1eae949" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33893 } } peers { permanent_uuid: "c9be399be7f1481a8131347c2ab2fa8b" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38137 } } peers { permanent_uuid: "64abfceb9020451192fd2e81881aee93" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 37335 } }
I20260927 07:03:53.612378 29222 leader_election.cc:288] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893), c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137)
W20260927 07:03:53.613336 28798 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c1aac6ee35417f82cc6bada1eae949 (127.13.121.195:33893): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
W20260927 07:03:53.613493 28795 leader_election.cc:341] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9be399be7f1481a8131347c2ab2fa8b (127.13.121.194:38137): Not found: Tablet not found: 3aaef909438c4ea489fd40c0ba27d6e9
I20260927 07:03:53.613652 28795 leader_election.cc:302] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64abfceb9020451192fd2e81881aee93; no voters: 79c1aac6ee35417f82cc6bada1eae949, c9be399be7f1481a8131347c2ab2fa8b
I20260927 07:03:53.613888 29222 raft_consensus.cc:2726] T 3aaef909438c4ea489fd40c0ba27d6e9 P 64abfceb9020451192fd2e81881aee93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:03:53.732605 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:53.731731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:45081)
0927 07:03:53.732237 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:53.732358 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":255,"threads_started":1}
I20260927 07:03:54.171434 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:54.170546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:60623)
0927 07:03:54.171100 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:54.171239 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":311,"threads_started":1}
I20260927 07:03:54.608402 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:54.607636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:43471)
0927 07:03:54.608113 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:54.608215 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":256,"threads_started":1}
I20260927 07:03:54.734645 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:54.733879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.193:36267)
0927 07:03:54.734333 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:54.734433 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":220,"threads_started":1}
I20260927 07:03:55.173382 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:55.172589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.194:59071)
0927 07:03:55.173100 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:55.173212 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":243,"threads_started":1}
I20260927 07:03:55.583684 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 28745
I20260927 07:03:55.598896 29228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:55.584761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37344 (local address 127.13.121.193:37335)
0927 07:03:55.585379 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:55.585389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:55.585420 (+ 31us) server_negotiation.cc:407] Connection header received
0927 07:03:55.585488 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:55.585494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:55.585751 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:55.585926 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:55.587950 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:55.588882 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:55.590174 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:55.590758 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:55.590879 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:55.593436 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:55.593449 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:55.593465 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:55.593531 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:55.596322 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:55.596990 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:55.596998 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:55.597000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:55.597126 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:55.597421 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:55.597424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:55.597426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:55.597804 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:55.598033 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:55.598311 (+ 278us) server_negotiation.cc:299] Negotiation successful
0927 07:03:55.598539 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":280,"threads_started":1}
I20260927 07:03:55.600231 28904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:55.610342 29230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:55.609624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34703 (local address 127.13.121.195:60873)
0927 07:03:55.610079 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:55.610188 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":223,"threads_started":1}
I20260927 07:03:55.688747 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 28745
I20260927 07:03:55.703850 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 28912
I20260927 07:03:55.716710 29231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:55.704551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60030 (local address 127.13.121.194:38137)
0927 07:03:55.704969 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:55.704974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:55.704990 (+ 16us) server_negotiation.cc:407] Connection header received
0927 07:03:55.705045 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:55.705049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:55.705107 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:55.705192 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:55.706466 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:55.707275 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:55.708472 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:55.708914 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:55.709000 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:55.711564 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:55.711569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:55.711572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:55.711607 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:55.714213 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:55.714949 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:55.714953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:55.714955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:55.715088 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:55.715428 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:55.715431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:55.715433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:55.715718 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:55.715913 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:55.716138 (+ 225us) server_negotiation.cc:299] Negotiation successful
0927 07:03:55.716321 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":214,"threads_started":1}
I20260927 07:03:55.717499 29036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:55.800318 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 28912
I20260927 07:03:55.814416 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 29044
I20260927 07:03:55.827520 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:55.815226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.13.121.195:33893)
0927 07:03:55.815735 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:55.815739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:55.815755 (+ 16us) server_negotiation.cc:407] Connection header received
0927 07:03:55.815808 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:55.815811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:55.815877 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:55.815976 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:55.817385 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:55.818134 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:55.819408 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:55.819851 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:55.819960 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:55.822423 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:55.822428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:55.822431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:55.822464 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:55.824962 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:55.825620 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:55.825624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:55.825627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:55.825764 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:55.826093 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:55.826097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:55.826099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:55.826374 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:55.826620 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:55.826908 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 07:03:55.827165 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":244,"threads_started":1}
I20260927 07:03:55.828266 29167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:55.910398 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 29044
2026-09-27T07:03:55Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (23984 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:03:56 dist-test-slave-p6hz krb5kdc[29237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:03:56 dist-test-slave-p6hz krb5kdc[29237](info): set up 2 sockets
Sep 27 07:03:56 dist-test-slave-p6hz krb5kdc[29237](info): commencing operation
krb5kdc: starting...
W20260927 07:03:56.931586 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.003s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:03:57 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:03:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:03:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:03:57.163810 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38269
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:57.493149 29250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:57.493453 29250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:57.493570 29250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:57.500991 29250 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 07:03:57.501098 29250 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 07:03:57.501145 29250 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 07:03:57.501214 29250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:03:57.501283 29250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:57.501376 29250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:57.501430 29250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:03:57.501506 29250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:03:57.511646 29250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:57.512933 29250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:57.514372 29250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:57.526710 29255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:57.526774 29250 server_base.cc:1034] running on GCE node
W20260927 07:03:57.526758 29256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:57.526943 29258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:57.527920 29250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:57.529353 29250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:57.530578 29250 hybrid_clock.cc:648] HybridClock initialized: now 1790492637530514 us; error 41 us; skew 500 ppm
I20260927 07:03:57.530915 29250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:57 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492637, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:57.541004 29250 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:03:57.542608 29250 webserver.cc:460] Webserver started at http://127.13.121.254:38569/ using document root <none> and password file <none>
I20260927 07:03:57.543433 29250 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:57.543563 29250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:57.543862 29250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:57.546900 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "63e3f2e2e5b14940bcefa3b7530226ea"
format_stamp: "Formatted at 2026-09-27 07:03:57 on dist-test-slave-p6hz"
server_key: "ce46c426a280bec4843cc4afe652e47c"
server_key_iv: "6aca60c991aafffc3a2decbcb2412509"
server_key_version: "encryptionkey@0"
I20260927 07:03:57.547681 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "63e3f2e2e5b14940bcefa3b7530226ea"
format_stamp: "Formatted at 2026-09-27 07:03:57 on dist-test-slave-p6hz"
server_key: "ce46c426a280bec4843cc4afe652e47c"
server_key_iv: "6aca60c991aafffc3a2decbcb2412509"
server_key_version: "encryptionkey@0"
I20260927 07:03:57.552929 29250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:03:57.556344 29265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:57.557662 29250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:03:57.557826 29250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "63e3f2e2e5b14940bcefa3b7530226ea"
format_stamp: "Formatted at 2026-09-27 07:03:57 on dist-test-slave-p6hz"
server_key: "ce46c426a280bec4843cc4afe652e47c"
server_key_iv: "6aca60c991aafffc3a2decbcb2412509"
server_key_version: "encryptionkey@0"
I20260927 07:03:57.558034 29250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:57.587230 29250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:57.591496 29250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:57.591765 29250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:57.609525 29250 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:38269
I20260927 07:03:57.609545 29316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:38269 every 8 connection(s)
I20260927 07:03:57.610870 29250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:03:57.614602 29317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:57.616290 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 29250
I20260927 07:03:57.616530 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:03:57.617344 13799 external_mini_cluster.cc:1428] Setting key e46cee0c88aa94eeae16ee85cc78ce56
I20260927 07:03:57.627334 29317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Bootstrap starting.
I20260927 07:03:57.631278 29317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Neither blocks nor log segments found. Creating new log.
Sep 27 07:03:57 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:57.632498 29317 log.cc:824] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:57.635689 29317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: No bootstrap required, opened a new log
I20260927 07:03:57.640726 29320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:57.619456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.13.121.254:38269)
0927 07:03:57.620290 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:57.620304 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:57.620345 (+ 41us) server_negotiation.cc:407] Connection header received
0927 07:03:57.622077 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:57.622103 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:57.622489 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:57.622806 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:57.624672 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:57.626020 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:57.628047 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:57.628736 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:57.628879 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:57.633678 (+ 4799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:57.633690 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:57.633711 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:57.633762 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:57.636658 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:57.637549 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:57.637555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:57.637564 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:57.637734 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:57.638112 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:57.638115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:57.638117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:57.638676 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:57.638916 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:57.639365 (+ 449us) server_negotiation.cc:299] Negotiation successful
0927 07:03:57.639715 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":384,"threads_started":1}
I20260927 07:03:57.644146 29317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:03:57.644498 29317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:57.644605 29317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e3f2e2e5b14940bcefa3b7530226ea, State: Initialized, Role: FOLLOWER
I20260927 07:03:57.645140 29317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:03:57.645352 29317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:03:57.645474 29317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:03:57.645612 29317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:57.647794 29317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:03:57.648276 29317 leader_election.cc:302] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63e3f2e2e5b14940bcefa3b7530226ea; no voters:
I20260927 07:03:57.648808 29317 leader_election.cc:288] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:03:57.648977 29322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:03:57.649946 29322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 1 LEADER]: Becoming Leader. State: Replica: 63e3f2e2e5b14940bcefa3b7530226ea, State: Running, Role: LEADER
I20260927 07:03:57.650504 29322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:03:57.651026 29317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:03:57.653584 29323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } } }
I20260927 07:03:57.654004 29323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:57.654479 29324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63e3f2e2e5b14940bcefa3b7530226ea. Latest consensus state: current_term: 1 leader_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } } }
I20260927 07:03:57.654865 29324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: This master's current role is: LEADER
I20260927 07:03:57.656020 29328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:03:57.661861 29328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:03:57.671190 29328 catalog_manager.cc:1347] Generated new cluster ID: f9d6933dde764e1c89649558d4a09369
I20260927 07:03:57.671306 29328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:03:57.719578 29328 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:03:57.721371 29328 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:03:57.736539 29328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Generated new TSK 0
I20260927 07:03:57.737450 29328 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:03:57.798942 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38269
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:58.126849 29343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:58.127362 29343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:58.127477 29343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:58.134907 29343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:58.135028 29343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:58.135182 29343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:03:58.145750 29343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:58.146996 29343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:58.148284 29343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:58.162410 29349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:58.162344 29351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:58.163067 29348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:58.165025 29343 server_base.cc:1034] running on GCE node
I20260927 07:03:58.165741 29343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:58.166779 29343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:58.167985 29343 hybrid_clock.cc:648] HybridClock initialized: now 1790492638167933 us; error 40 us; skew 500 ppm
I20260927 07:03:58.168295 29343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:58 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492638, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:58.171581 29343 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:03:58.173107 29343 webserver.cc:460] Webserver started at http://127.13.121.193:45579/ using document root <none> and password file <none>
I20260927 07:03:58.173818 29343 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:58.173942 29343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:58.174242 29343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:58.177157 29343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "4c9c527218ce464da8a12134a73ea6cb"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1f5377e3840bd51730c5286fecc43abd"
server_key_iv: "3e6f7dd07f7ed8a71562fa875b15f314"
server_key_version: "encryptionkey@0"
I20260927 07:03:58.177891 29343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "4c9c527218ce464da8a12134a73ea6cb"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1f5377e3840bd51730c5286fecc43abd"
server_key_iv: "3e6f7dd07f7ed8a71562fa875b15f314"
server_key_version: "encryptionkey@0"
I20260927 07:03:58.183269 29343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260927 07:03:58.186942 29358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:58.188433 29343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:03:58.188639 29343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "4c9c527218ce464da8a12134a73ea6cb"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1f5377e3840bd51730c5286fecc43abd"
server_key_iv: "3e6f7dd07f7ed8a71562fa875b15f314"
server_key_version: "encryptionkey@0"
I20260927 07:03:58.188848 29343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:58.200454 29343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:58.204313 29343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:58.204598 29343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:58.205542 29343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:58.206915 29343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:58.206995 29343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:58.207108 29343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:58.207175 29343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260927 07:03:58.248183 29343 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:44001
I20260927 07:03:58.248232 29470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:44001 every 8 connection(s)
I20260927 07:03:58.249604 29343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:03:58.257398 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 29343
I20260927 07:03:58.257629 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:03:58.258212 13799 external_mini_cluster.cc:1428] Setting key 35795dc9ae21ff3d1aef0245c6ee1097
Sep 27 07:03:58 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492638, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:58.269982 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:58.252274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:55995 (local address 127.13.121.254:38269)
0927 07:03:58.252716 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:58.252721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:58.253732 (+ 1011us) server_negotiation.cc:407] Connection header received
0927 07:03:58.255615 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:58.255621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:58.255702 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:58.255797 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:58.257538 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:58.258321 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:58.260265 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:58.260676 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:58.260788 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:58.264510 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:58.264516 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:58.264519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:58.264553 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:58.266901 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:58.267694 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:58.267700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:58.267703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:58.267845 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:58.268257 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:58.268261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:58.268264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:58.268602 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:58.268746 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:58.269410 (+ 664us) server_negotiation.cc:299] Negotiation successful
0927 07:03:58.269639 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":207,"threads_started":1}
I20260927 07:03:58.270871 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:58.252847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:55995)
0927 07:03:58.253557 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:58.253599 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:58.255328 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:58.256037 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:58.256046 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:58.256587 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:58.257311 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:58.257338 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:58.258497 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:58.258502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:58.260114 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:58.260138 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:58.260897 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:58.260904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:58.261147 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:58.261899 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:58.261923 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:58.264270 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:58.267044 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:58.267054 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:58.267073 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:58.267553 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:58.267953 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:58.267957 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:58.267960 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:58.268152 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:58.268703 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:58.268708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:58.269035 (+ 327us) client_negotiation.cc:769] Sending connection context
0927 07:03:58.269269 (+ 234us) client_negotiation.cc:240] Negotiation successful
0927 07:03:58.269700 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":276,"threads_started":1}
I20260927 07:03:58.272450 29471 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38269
I20260927 07:03:58.272742 29471 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:58.273429 29471 heartbeater.cc:511] Master 127.13.121.254:38269 requested a full tablet report, sending...
I20260927 07:03:58.275501 29282 ts_manager.cc:194] Registered new tserver with Master: 4c9c527218ce464da8a12134a73ea6cb (127.13.121.193:44001)
I20260927 07:03:58.276932 29282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:55995
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:03:58.311184 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38269
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:58.643767 29477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:58.644050 29477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:58.644145 29477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:58.651355 29477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:58.651473 29477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:58.651595 29477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:03:58.661546 29477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:58.663094 29477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:58.664570 29477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:58.677597 29482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:58.677629 29483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:58.677724 29485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:58.678422 29477 server_base.cc:1034] running on GCE node
I20260927 07:03:58.679080 29477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:58.680107 29477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:58.681345 29477 hybrid_clock.cc:648] HybridClock initialized: now 1790492638681286 us; error 50 us; skew 500 ppm
I20260927 07:03:58.681725 29477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:58 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492638, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:58.685092 29477 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:03:58.686832 29477 webserver.cc:460] Webserver started at http://127.13.121.194:42581/ using document root <none> and password file <none>
I20260927 07:03:58.687628 29477 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:58.687772 29477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:58.688089 29477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:58.691505 29477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "947c8106699644f0b5058c2ea438aa2c"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1a083542fe069452ade0a5a9d3f27a84"
server_key_iv: "72596f6cfed49f81b4282ea2655fb61f"
server_key_version: "encryptionkey@0"
I20260927 07:03:58.692485 29477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "947c8106699644f0b5058c2ea438aa2c"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1a083542fe069452ade0a5a9d3f27a84"
server_key_iv: "72596f6cfed49f81b4282ea2655fb61f"
server_key_version: "encryptionkey@0"
I20260927 07:03:58.698089 29477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:03:58.702276 29492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:58.703943 29477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:03:58.704140 29477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "947c8106699644f0b5058c2ea438aa2c"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1a083542fe069452ade0a5a9d3f27a84"
server_key_iv: "72596f6cfed49f81b4282ea2655fb61f"
server_key_version: "encryptionkey@0"
I20260927 07:03:58.704346 29477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:58.722903 29477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:58.727705 29477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:58.727998 29477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:58.729696 29477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:58.731266 29477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:58.731354 29477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:58.731490 29477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:58.731542 29477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:58.762431 29477 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:42677
I20260927 07:03:58.762472 29604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:42677 every 8 connection(s)
I20260927 07:03:58.763769 29477 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:03:58.773535 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 29477
I20260927 07:03:58.773713 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:03:58.774173 13799 external_mini_cluster.cc:1428] Setting key 30221f68d42cbe7887ca8f83f9d850ae
Sep 27 07:03:58 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492638, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:58.783746 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:58.766172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:54903 (local address 127.13.121.254:38269)
0927 07:03:58.766336 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:58.766340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:58.767556 (+ 1216us) server_negotiation.cc:407] Connection header received
0927 07:03:58.769392 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:58.769398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:58.769488 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:58.769586 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:58.771216 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:58.771921 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:58.773543 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:58.774025 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:58.774145 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:58.778107 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:58.778112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:58.778115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:58.778151 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:58.780606 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:58.781375 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:58.781380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:58.781382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:58.781526 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:58.781947 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:58.781957 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:58.781960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:58.782247 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:58.782384 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:58.783110 (+ 726us) server_negotiation.cc:299] Negotiation successful
0927 07:03:58.783386 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 07:03:58.784945 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:58.766585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:54903)
0927 07:03:58.767379 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:58.767422 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:58.769091 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:58.769850 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:58.769860 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:58.770400 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:58.770995 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:58.771013 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:58.772086 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:58.772090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:58.773378 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:58.773392 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:58.774259 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:58.774267 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:58.774502 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:58.775294 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:58.775334 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:58.777858 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:58.780762 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:58.780773 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:58.780792 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:58.781246 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:58.781636 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:58.781641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:58.781645 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:58.781833 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:58.782373 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:58.782382 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:58.782747 (+ 365us) client_negotiation.cc:769] Sending connection context
0927 07:03:58.783000 (+ 253us) client_negotiation.cc:240] Negotiation successful
0927 07:03:58.783620 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":305,"threads_started":1}
I20260927 07:03:58.786473 29605 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38269
I20260927 07:03:58.786787 29605 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:58.787649 29605 heartbeater.cc:511] Master 127.13.121.254:38269 requested a full tablet report, sending...
I20260927 07:03:58.789314 29282 ts_manager.cc:194] Registered new tserver with Master: 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677)
I20260927 07:03:58.790325 29282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:54903
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:03:58.826627 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38269
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:03:59.150446 29610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:03:59.150776 29610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:03:59.150923 29610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:03:59.158947 29610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:03:59.159111 29610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:03:59.159314 29610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:03:59.169530 29610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:03:59.170874 29610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:03:59.172223 29610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:03:59.185024 29618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:59.185164 29616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:03:59.185190 29615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:03:59.185640 29610 server_base.cc:1034] running on GCE node
I20260927 07:03:59.186358 29610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:03:59.187397 29610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:03:59.188606 29610 hybrid_clock.cc:648] HybridClock initialized: now 1790492639188556 us; error 41 us; skew 500 ppm
I20260927 07:03:59.188943 29610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:03:59 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492639, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:03:59.192071 29610 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:03:59.193569 29610 webserver.cc:460] Webserver started at http://127.13.121.195:33633/ using document root <none> and password file <none>
I20260927 07:03:59.194271 29610 fs_manager.cc:359] Metadata directory not provided
I20260927 07:03:59.194376 29610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:03:59.194681 29610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:03:59.197631 29610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "cfda194b51194355b9133bd3263bd255"
format_stamp: "Formatted at 2026-09-27 07:03:59 on dist-test-slave-p6hz"
server_key: "c5aefe99c67f8c3d50e589524d6ef759"
server_key_iv: "252438f659912577561e8b0f4e391b1f"
server_key_version: "encryptionkey@0"
I20260927 07:03:59.198320 29610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "cfda194b51194355b9133bd3263bd255"
format_stamp: "Formatted at 2026-09-27 07:03:59 on dist-test-slave-p6hz"
server_key: "c5aefe99c67f8c3d50e589524d6ef759"
server_key_iv: "252438f659912577561e8b0f4e391b1f"
server_key_version: "encryptionkey@0"
I20260927 07:03:59.203347 29610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260927 07:03:59.206833 29625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:59.208324 29610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:03:59.208461 29610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "cfda194b51194355b9133bd3263bd255"
format_stamp: "Formatted at 2026-09-27 07:03:59 on dist-test-slave-p6hz"
server_key: "c5aefe99c67f8c3d50e589524d6ef759"
server_key_iv: "252438f659912577561e8b0f4e391b1f"
server_key_version: "encryptionkey@0"
I20260927 07:03:59.208632 29610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:03:59.226146 29610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:03:59.230571 29610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:03:59.230813 29610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:03:59.231801 29610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:03:59.233201 29610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:03:59.233273 29610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:59.233371 29610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:03:59.233413 29610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:03:59.263746 29610 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:45665
I20260927 07:03:59.263787 29737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:45665 every 8 connection(s)
I20260927 07:03:59.265015 29610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:03:59.268767 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 29610
I20260927 07:03:59.268962 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:03:59.269515 13799 external_mini_cluster.cc:1428] Setting key ef84d4b3ec55a6177acfa3786744dd73
Sep 27 07:03:59 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492639, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:03:59.280627 29471 heartbeater.cc:503] Master 127.13.121.254:38269 was elected leader, sending a full tablet report...
I20260927 07:03:59.286850 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.267476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:43947 (local address 127.13.121.254:38269)
0927 07:03:59.267665 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.267670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.268976 (+ 1306us) server_negotiation.cc:407] Connection header received
0927 07:03:59.271254 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.271260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.271349 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.271442 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:59.273124 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.273846 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.275395 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.275879 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.276010 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.279602 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:59.279607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:59.279611 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:59.279650 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:59.282788 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.283944 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.283949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.283953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.284127 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.284674 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.284679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.284682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.285061 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:59.285211 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.286364 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.286577 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 07:03:59.287986 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.267939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:43947)
0927 07:03:59.268706 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:59.268757 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:59.270918 (+ 2161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:59.271718 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:59.271728 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:59.272327 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:03:59.272975 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.272993 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.273959 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.273962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.275243 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.275255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.276191 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.276197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.276428 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.277050 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:03:59.277086 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:03:59.279343 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:03:59.282940 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:59.282950 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:59.282970 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:59.283712 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:59.284268 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:03:59.284273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:03:59.284276 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:03:59.284538 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:03:59.285309 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:03:59.285317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:03:59.285864 (+ 547us) client_negotiation.cc:769] Sending connection context
0927 07:03:59.286214 (+ 350us) client_negotiation.cc:240] Negotiation successful
0927 07:03:59.286722 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":261,"threads_started":1}
I20260927 07:03:59.289508 29738 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38269
I20260927 07:03:59.289809 29738 heartbeater.cc:464] Registering TS with master...
I20260927 07:03:59.290519 29738 heartbeater.cc:511] Master 127.13.121.254:38269 requested a full tablet report, sending...
I20260927 07:03:59.292083 29282 ts_manager.cc:194] Registered new tserver with Master: cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665)
I20260927 07:03:59.292924 29282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:43947
I20260927 07:03:59.297459 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:03:59.313997 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.301789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51138 (local address 127.13.121.254:38269)
0927 07:03:59.301960 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.301967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.302324 (+ 357us) server_negotiation.cc:407] Connection header received
0927 07:03:59.302496 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.302500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.302564 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.302646 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:59.304070 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.304813 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.306162 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.306564 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.306675 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.309026 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:59.309031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:59.309035 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:59.309072 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:59.311390 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.312206 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.312210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.312212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.312347 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.312756 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.312760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.312762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.313139 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:59.313342 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.313586 (+ 244us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.313764 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:03:59.318082 29282 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:03:59.325146 29282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:03:59.348551 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.337482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:44001 (local address 127.0.0.1:46350)
0927 07:03:59.338554 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:59.338595 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:59.338849 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:59.339747 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:59.339755 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:59.339792 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:59.340249 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.340262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.342602 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.342609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.345262 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.345283 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.347420 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.347428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.347688 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.347754 (+ 66us) client_negotiation.cc:769] Sending connection context
0927 07:03:59.347880 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 07:03:59.348124 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"thread_start_us":620,"threads_started":1}
I20260927 07:03:59.350109 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.337830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.13.121.193:44001)
0927 07:03:59.339063 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.339072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.339113 (+ 41us) server_negotiation.cc:407] Connection header received
0927 07:03:59.339239 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.339246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.339500 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.340459 (+ 959us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:59.340699 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.342428 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.345522 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.347191 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.349234 (+ 2043us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.349467 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.349544 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.349744 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1071,"thread_start_us":938,"threads_started":1}
I20260927 07:03:59.356382 29406 tablet_service.cc:1467] Processing CreateTablet for tablet 15e3f24cf518496fabd69eee8ef3637a (DEFAULT_TABLE table=test-table [id=e7c2e9700ae944e9ad62eea1a784b064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:59.358146 29406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e3f24cf518496fabd69eee8ef3637a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:59.358470 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.339538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42677 (local address 127.0.0.1:57270)
0927 07:03:59.348836 (+ 9298us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:59.348858 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:59.348977 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:59.352503 (+ 3526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:59.352508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:59.352539 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:59.352722 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.352735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.354402 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.354408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.355775 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.355788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.357651 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.357662 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.357869 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.357904 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 07:03:59.358020 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 07:03:59.358170 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9188,"thread_start_us":12300,"threads_started":1}
I20260927 07:03:59.358563 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.340022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.13.121.194:42677)
0927 07:03:59.351738 (+ 11716us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.351750 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.351790 (+ 40us) server_negotiation.cc:407] Connection header received
0927 07:03:59.351922 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.351932 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.352244 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.352554 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:59.352863 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.354205 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.355932 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.357455 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.357672 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.357848 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.358056 (+ 208us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.358235 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11514,"thread_start_us":11331,"threads_started":1}
I20260927 07:03:59.360414 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.338925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:45665 (local address 127.0.0.1:41474)
0927 07:03:59.339440 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:59.339457 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:59.339551 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:59.353051 (+ 13500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:59.353057 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:59.353085 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:59.353446 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.353460 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.356092 (+ 2632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.356098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.358008 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.358023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.359641 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.359648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.359815 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.359846 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 07:03:59.359932 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 07:03:59.360072 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":286,"threads_started":1}
I20260927 07:03:59.361426 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.339416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.13.121.195:45665)
0927 07:03:59.352467 (+ 13051us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.352476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.352506 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:03:59.352600 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.352608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.352823 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.353648 (+ 825us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:59.353951 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.355949 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.358196 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.359440 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.359653 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.359820 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.360939 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.361118 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12874,"thread_start_us":12732,"threads_started":1}
I20260927 07:03:59.362896 29540 tablet_service.cc:1467] Processing CreateTablet for tablet 15e3f24cf518496fabd69eee8ef3637a (DEFAULT_TABLE table=test-table [id=e7c2e9700ae944e9ad62eea1a784b064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:59.364363 29540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e3f24cf518496fabd69eee8ef3637a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:59.371438 29673 tablet_service.cc:1467] Processing CreateTablet for tablet 15e3f24cf518496fabd69eee8ef3637a (DEFAULT_TABLE table=test-table [id=e7c2e9700ae944e9ad62eea1a784b064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:03:59.373077 29673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e3f24cf518496fabd69eee8ef3637a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:03:59.377931 29757 tablet_bootstrap.cc:492] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: Bootstrap starting.
I20260927 07:03:59.380173 29756 tablet_bootstrap.cc:492] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: Bootstrap starting.
I20260927 07:03:59.381942 29757 tablet_bootstrap.cc:654] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:59.383790 29756 tablet_bootstrap.cc:654] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:59.384104 29757 log.cc:824] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:59.384992 29756 log.cc:824] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:59.387528 29757 tablet_bootstrap.cc:492] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: No bootstrap required, opened a new log
I20260927 07:03:59.387528 29760 tablet_bootstrap.cc:492] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: Bootstrap starting.
I20260927 07:03:59.387830 29757 ts_tablet_manager.cc:1397] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260927 07:03:59.390949 29756 tablet_bootstrap.cc:492] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: No bootstrap required, opened a new log
I20260927 07:03:59.390933 29760 tablet_bootstrap.cc:654] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: Neither blocks nor log segments found. Creating new log.
I20260927 07:03:59.391273 29756 ts_tablet_manager.cc:1397] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.005s
I20260927 07:03:59.392171 29760 log.cc:824] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: Log is configured to *not* fsync() on all Append() calls
I20260927 07:03:59.395257 29760 tablet_bootstrap.cc:492] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: No bootstrap required, opened a new log
I20260927 07:03:59.395519 29760 ts_tablet_manager.cc:1397] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260927 07:03:59.396078 29757 raft_consensus.cc:348] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } }
I20260927 07:03:59.396446 29757 raft_consensus.cc:374] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:59.396534 29757 raft_consensus.cc:729] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947c8106699644f0b5058c2ea438aa2c, State: Initialized, Role: FOLLOWER
I20260927 07:03:59.397145 29757 consensus_queue.cc:260] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } }
I20260927 07:03:59.398180 29605 heartbeater.cc:503] Master 127.13.121.254:38269 was elected leader, sending a full tablet report...
I20260927 07:03:59.398898 29757 ts_tablet_manager.cc:1428] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260927 07:03:59.401531 29756 raft_consensus.cc:348] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } }
I20260927 07:03:59.402019 29756 raft_consensus.cc:374] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:59.402120 29756 raft_consensus.cc:729] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9c527218ce464da8a12134a73ea6cb, State: Initialized, Role: FOLLOWER
I20260927 07:03:59.402782 29756 consensus_queue.cc:260] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } }
I20260927 07:03:59.404675 29756 ts_tablet_manager.cc:1428] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260927 07:03:59.405610 29760 raft_consensus.cc:348] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } }
I20260927 07:03:59.405961 29760 raft_consensus.cc:374] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:03:59.406077 29760 raft_consensus.cc:729] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfda194b51194355b9133bd3263bd255, State: Initialized, Role: FOLLOWER
I20260927 07:03:59.406620 29760 consensus_queue.cc:260] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } }
I20260927 07:03:59.407760 29738 heartbeater.cc:503] Master 127.13.121.254:38269 was elected leader, sending a full tablet report...
I20260927 07:03:59.408387 29760 ts_tablet_manager.cc:1428] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: Time spent starting tablet: real 0.013s user 0.002s sys 0.010s
I20260927 07:03:59.410271 29763 raft_consensus.cc:482] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:03:59.410552 29763 raft_consensus.cc:504] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } }
I20260927 07:03:59.412251 29763 leader_election.cc:288] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665), 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677)
I20260927 07:03:59.420416 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.413464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:59075 (local address 127.13.121.194:42677)
0927 07:03:59.413651 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.413656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.414007 (+ 351us) server_negotiation.cc:407] Connection header received
0927 07:03:59.414165 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.414168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.414229 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.414306 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:59.414772 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.415807 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.418112 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.419368 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.419519 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.419595 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.420016 (+ 421us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.420155 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 07:03:59.420415 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.412581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:36611 (local address 127.13.121.195:45665)
0927 07:03:59.412776 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.412782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.413194 (+ 412us) server_negotiation.cc:407] Connection header received
0927 07:03:59.413478 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.413483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.413558 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.413630 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:03:59.414422 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.415545 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.418111 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.419358 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.419532 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.419596 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.420010 (+ 414us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.420163 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 07:03:59.420879 29765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.412466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:45665 (local address 127.13.121.193:36611)
0927 07:03:59.413096 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:59.413116 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:59.413306 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:59.413833 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:59.413838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:59.413864 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:59.414272 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.414283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.415665 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.415669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.417929 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.417946 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.419640 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.419646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.419816 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.419846 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 07:03:59.419948 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 07:03:59.420129 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":64,"spinlock_wait_cycles":24448,"thread_start_us":314,"threads_started":1}
I20260927 07:03:59.420898 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.413158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42677 (local address 127.13.121.193:59075)
0927 07:03:59.413891 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0927 07:03:59.413907 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:03:59.414041 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:03:59.414338 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:03:59.414342 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:03:59.414368 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:03:59.414637 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.414644 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.415926 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.415931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.417951 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:03:59.417961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.419599 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.419606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:03:59.419803 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.419843 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 07:03:59.419953 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 07:03:59.420130 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":27,"thread_start_us":243,"threads_started":1}
I20260927 07:03:59.421933 29560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e3f24cf518496fabd69eee8ef3637a" candidate_uuid: "4c9c527218ce464da8a12134a73ea6cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947c8106699644f0b5058c2ea438aa2c" is_pre_election: true
I20260927 07:03:59.421931 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e3f24cf518496fabd69eee8ef3637a" candidate_uuid: "4c9c527218ce464da8a12134a73ea6cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfda194b51194355b9133bd3263bd255" is_pre_election: true
I20260927 07:03:59.422412 29693 raft_consensus.cc:2454] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9c527218ce464da8a12134a73ea6cb in term 0.
I20260927 07:03:59.422412 29560 raft_consensus.cc:2454] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9c527218ce464da8a12134a73ea6cb in term 0.
I20260927 07:03:59.423082 29360 leader_election.cc:302] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9c527218ce464da8a12134a73ea6cb, 947c8106699644f0b5058c2ea438aa2c; no voters:
I20260927 07:03:59.423578 29763 raft_consensus.cc:2781] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:03:59.423743 29763 raft_consensus.cc:482] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:03:59.423931 29763 raft_consensus.cc:3037] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:59.426155 29763 raft_consensus.cc:504] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } }
I20260927 07:03:59.426828 29763 leader_election.cc:288] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 1 election: Requested vote from peers cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665), 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677)
I20260927 07:03:59.427348 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e3f24cf518496fabd69eee8ef3637a" candidate_uuid: "4c9c527218ce464da8a12134a73ea6cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfda194b51194355b9133bd3263bd255"
I20260927 07:03:59.427409 29560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e3f24cf518496fabd69eee8ef3637a" candidate_uuid: "4c9c527218ce464da8a12134a73ea6cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947c8106699644f0b5058c2ea438aa2c"
I20260927 07:03:59.427599 29560 raft_consensus.cc:3037] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:59.427589 29693 raft_consensus.cc:3037] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:03:59.429814 29693 raft_consensus.cc:2454] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9c527218ce464da8a12134a73ea6cb in term 1.
I20260927 07:03:59.429867 29560 raft_consensus.cc:2454] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9c527218ce464da8a12134a73ea6cb in term 1.
I20260927 07:03:59.430289 29360 leader_election.cc:302] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9c527218ce464da8a12134a73ea6cb, cfda194b51194355b9133bd3263bd255; no voters:
I20260927 07:03:59.430637 29763 raft_consensus.cc:2781] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:03:59.431097 29763 raft_consensus.cc:686] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 1 LEADER]: Becoming Leader. State: Replica: 4c9c527218ce464da8a12134a73ea6cb, State: Running, Role: LEADER
I20260927 07:03:59.431699 29763 consensus_queue.cc:237] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } }
I20260927 07:03:59.436524 29282 catalog_manager.cc:5495] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c9c527218ce464da8a12134a73ea6cb (127.13.121.193). New cstate: current_term: 1 leader_uuid: "4c9c527218ce464da8a12134a73ea6cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } health_report { overall_health: UNKNOWN } } }
I20260927 07:03:59.457015 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.451637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46352 (local address 127.13.121.193:44001)
0927 07:03:59.451847 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.451853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.451872 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:03:59.451988 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.451993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.452075 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.452206 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:03:59.452602 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.453332 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.454752 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.455214 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.455320 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.455883 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:03:59.456406 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:03:59.456558 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.456665 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.456755 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:03:59.465893 29560 raft_consensus.cc:1261] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [term 1 FOLLOWER]: Refusing update from remote peer 4c9c527218ce464da8a12134a73ea6cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:59.465893 29693 raft_consensus.cc:1261] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [term 1 FOLLOWER]: Refusing update from remote peer 4c9c527218ce464da8a12134a73ea6cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:03:59.467500 29767 consensus_queue.cc:1035] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:03:59.467868 29763 consensus_queue.cc:1035] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:03:59.481587 29771 mvcc.cc:205] Tried to move back new op lower bound from 7333857851240742912 to 7333857851116392448. Current Snapshot: MvccSnapshot[applied={T|T < 7333857851240742912}]
I20260927 07:03:59.502900 29282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:03:59.503181 29282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:03:59.506675 29282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Sending DeleteTablet for 3 replicas of tablet 15e3f24cf518496fabd69eee8ef3637a
I20260927 07:03:59.508184 29540 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e3f24cf518496fabd69eee8ef3637a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:59 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:57270
I20260927 07:03:59.508299 29673 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e3f24cf518496fabd69eee8ef3637a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:59 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:41474
I20260927 07:03:59.508899 29406 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e3f24cf518496fabd69eee8ef3637a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:03:59 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:46350
I20260927 07:03:59.509209 29782 tablet_replica.cc:330] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: stopping tablet replica
I20260927 07:03:59.509306 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 29343
I20260927 07:03:59.509642 29782 raft_consensus.cc:2229] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:03:59.510145 29782 raft_consensus.cc:2258] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:59.510895 29781 tablet_replica.cc:330] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: stopping tablet replica
I20260927 07:03:59.511353 29781 raft_consensus.cc:2229] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:03:59.511873 29781 raft_consensus.cc:2258] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:59.514322 29781 ts_tablet_manager.cc:1905] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:59.517742 29782 ts_tablet_manager.cc:1905] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:59.522070 29781 ts_tablet_manager.cc:1918] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:59.522262 29781 log.cc:1196] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/15e3f24cf518496fabd69eee8ef3637a
I20260927 07:03:59.522797 29781 ts_tablet_manager.cc:1939] T 15e3f24cf518496fabd69eee8ef3637a P cfda194b51194355b9133bd3263bd255: Deleting consensus metadata
I20260927 07:03:59.523458 29783 tablet_replica.cc:330] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: stopping tablet replica
I20260927 07:03:59.524228 29783 raft_consensus.cc:2229] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:03:59.525008 29783 raft_consensus.cc:2258] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:03:59.525195 29267 catalog_manager.cc:4841] TS cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665): tablet 15e3f24cf518496fabd69eee8ef3637a (table test-table [id=e7c2e9700ae944e9ad62eea1a784b064]) successfully deleted
I20260927 07:03:59.527364 29783 ts_tablet_manager.cc:1905] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:03:59.527808 29782 ts_tablet_manager.cc:1918] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:59.527982 29782 log.cc:1196] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/15e3f24cf518496fabd69eee8ef3637a
I20260927 07:03:59.528493 29782 ts_tablet_manager.cc:1939] T 15e3f24cf518496fabd69eee8ef3637a P 947c8106699644f0b5058c2ea438aa2c: Deleting consensus metadata
I20260927 07:03:59.530373 29267 catalog_manager.cc:4841] TS 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677): tablet 15e3f24cf518496fabd69eee8ef3637a (table test-table [id=e7c2e9700ae944e9ad62eea1a784b064]) successfully deleted
Sep 27 07:03:59 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:03:59.534922 29783 ts_tablet_manager.cc:1918] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:03:59.535100 29783 log.cc:1196] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/15e3f24cf518496fabd69eee8ef3637a
I20260927 07:03:59.535565 29783 ts_tablet_manager.cc:1939] T 15e3f24cf518496fabd69eee8ef3637a P 4c9c527218ce464da8a12134a73ea6cb: Deleting consensus metadata
I20260927 07:03:59.537492 29267 catalog_manager.cc:4841] TS 4c9c527218ce464da8a12134a73ea6cb (127.13.121.193:44001): tablet 15e3f24cf518496fabd69eee8ef3637a (table test-table [id=e7c2e9700ae944e9ad62eea1a784b064]) successfully deleted
I20260927 07:03:59.542342 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.510062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.13.121.193:44001)
0927 07:03:59.510231 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.510237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.523832 (+ 13595us) server_negotiation.cc:407] Connection header received
0927 07:03:59.524134 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.524140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.524228 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.524306 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:59.526352 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.527472 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.529484 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.530210 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.530307 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.536033 (+ 5726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:59.536047 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:59.536068 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:59.536115 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:59.539414 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.540292 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.540299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.540301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.540453 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.540858 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.540861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.540863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.541208 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:59.541438 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.541849 (+ 411us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.542058 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 07:03:59.543004 29466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:59.640980 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 29343
I20260927 07:03:59.656432 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 29477
Sep 27 07:03:59 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:03:59.671345 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.657130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57274 (local address 127.13.121.194:42677)
0927 07:03:59.657372 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.657378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.657399 (+ 21us) server_negotiation.cc:407] Connection header received
0927 07:03:59.657511 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.657517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.657589 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.657674 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:59.658888 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.659760 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.661049 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.661479 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.661592 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.665840 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:59.665857 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:59.665874 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:59.665918 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:59.668594 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.669280 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.669288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.669290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.669422 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.669763 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.669767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.669769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.670160 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:59.670439 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.670769 (+ 330us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.670993 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260927 07:03:59.672017 29600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:59.761063 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 29477
I20260927 07:03:59.775045 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 29610
Sep 27 07:03:59 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:03:59.790081 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:03:59.775842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41484 (local address 127.13.121.195:45665)
0927 07:03:59.776032 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 07:03:59.776037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:03:59.776055 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:03:59.776113 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:03:59.776117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:03:59.776179 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:03:59.776262 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:03:59.777597 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.778356 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.779737 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:03:59.780165 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:03:59.780274 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:03:59.784493 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:03:59.784514 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:03:59.784530 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:03:59.784574 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:03:59.787066 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.787872 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.787880 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.787883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.788035 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:03:59.788422 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:03:59.788425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:03:59.788427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:03:59.788884 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:03:59.789170 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0927 07:03:59.789492 (+ 322us) server_negotiation.cc:299] Negotiation successful
0927 07:03:59.789714 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:03:59.790669 29733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:03:59.879895 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 29610
I20260927 07:03:59.893800 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 29250
I20260927 07:03:59.894945 29312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:04:00.007839 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 29250
I20260927 07:04:20.026268 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38269
--webserver_interface=127.13.121.254
--webserver_port=38569
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:04:20.448545 29789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:04:20.448982 29789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:04:20.449087 29789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:04:20.457325 29789 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 07:04:20.457477 29789 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 07:04:20.457520 29789 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 07:04:20.457567 29789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:04:20.457629 29789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:04:20.457715 29789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:04:20.457912 29789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:04:20.458063 29789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:04:20.469933 29789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=38569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:04:20.471954 29789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:04:20.473722 29789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:04:20.502573 29798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:20.502556 29795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:20.502553 29796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:04:20.509375 29789 server_base.cc:1034] running on GCE node
I20260927 07:04:20.510664 29789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:04:20.512894 29789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:04:20.514201 29789 hybrid_clock.cc:648] HybridClock initialized: now 1790492660514113 us; error 57 us; skew 500 ppm
I20260927 07:04:20.514701 29789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:04:20 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492660, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:04:20.524323 29789 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:04:20.526854 29789 webserver.cc:460] Webserver started at http://127.13.121.254:38569/ using document root <none> and password file <none>
I20260927 07:04:20.528051 29789 fs_manager.cc:359] Metadata directory not provided
I20260927 07:04:20.528226 29789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:04:20.537065 29789 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.001s
I20260927 07:04:20.541657 29805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:20.543999 29789 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 07:04:20.544306 29789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "63e3f2e2e5b14940bcefa3b7530226ea"
format_stamp: "Formatted at 2026-09-27 07:03:57 on dist-test-slave-p6hz"
server_key: "ce46c426a280bec4843cc4afe652e47c"
server_key_iv: "6aca60c991aafffc3a2decbcb2412509"
server_key_version: "encryptionkey@0"
I20260927 07:04:20.545742 29789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:04:20.585953 29789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:04:20.591952 29789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:04:20.592306 29789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:04:20.615572 29789 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:38269
I20260927 07:04:20.615585 29856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:38269 every 8 connection(s)
I20260927 07:04:20.617553 29789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:04:20.620174 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 29789
I20260927 07:04:20.621193 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:44001
--local_ip_for_outbound_sockets=127.13.121.193
--tserver_master_addrs=127.13.121.254:38269
--webserver_port=45579
--webserver_interface=127.13.121.193
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:04:20.627841 29857 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:04:20.632561 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Bootstrap starting.
I20260927 07:04:20.653548 29857 log.cc:824] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Log is configured to *not* fsync() on all Append() calls
I20260927 07:04:20.666337 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:04:20.666872 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Bootstrap complete.
I20260927 07:04:20.676378 29857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:04:20.677256 29857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e3f2e2e5b14940bcefa3b7530226ea, State: Initialized, Role: FOLLOWER
I20260927 07:04:20.677965 29857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:04:20.678208 29857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:04:20.678326 29857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:04:20.678489 29857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 1 FOLLOWER]: Advancing to term 2
I20260927 07:04:20.682555 29857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:04:20.683089 29857 leader_election.cc:302] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63e3f2e2e5b14940bcefa3b7530226ea; no voters:
I20260927 07:04:20.683768 29857 leader_election.cc:288] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 07:04:20.683920 29861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 2 FOLLOWER]: Leader election won for term 2
I20260927 07:04:20.685076 29861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 2 LEADER]: Becoming Leader. State: Replica: 63e3f2e2e5b14940bcefa3b7530226ea, State: Running, Role: LEADER
I20260927 07:04:20.685652 29861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:04:20.686147 29857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:04:20.688158 29863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63e3f2e2e5b14940bcefa3b7530226ea. Latest consensus state: current_term: 2 leader_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } } }
I20260927 07:04:20.688552 29863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: This master's current role is: LEADER
I20260927 07:04:20.689182 29862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } } }
I20260927 07:04:20.689494 29862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: This master's current role is: LEADER
I20260927 07:04:20.692094 29868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:04:20.701684 29868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:04:20.703353 29868 catalog_manager.cc:1259] Loaded cluster ID: f9d6933dde764e1c89649558d4a09369
I20260927 07:04:20.703471 29868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:04:20.708014 29868 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:04:20.709844 29868 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Loaded TSK: 0
I20260927 07:04:20.741497 29868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Generated new TSK 1
I20260927 07:04:20.741928 29868 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 07:04:21.000614 29859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:04:21.000929 29859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:04:21.001061 29859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:04:21.008459 29859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:04:21.008595 29859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:04:21.008749 29859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:04:21.018949 29859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:44001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=45579
--tserver_master_addrs=127.13.121.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:04:21.020334 29859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:04:21.021646 29859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:04:21.036278 29885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:21.036764 29884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:21.037223 29887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:04:21.037889 29859 server_base.cc:1034] running on GCE node
I20260927 07:04:21.038666 29859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:04:21.039758 29859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:04:21.040982 29859 hybrid_clock.cc:648] HybridClock initialized: now 1790492661040911 us; error 63 us; skew 500 ppm
I20260927 07:04:21.041299 29859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:04:21 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492661, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:04:21.044800 29859 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:04:21.046479 29859 webserver.cc:460] Webserver started at http://127.13.121.193:45579/ using document root <none> and password file <none>
I20260927 07:04:21.047297 29859 fs_manager.cc:359] Metadata directory not provided
I20260927 07:04:21.047463 29859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:04:21.053822 29859 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 07:04:21.057150 29894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:21.058866 29859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:04:21.059067 29859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "4c9c527218ce464da8a12134a73ea6cb"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1f5377e3840bd51730c5286fecc43abd"
server_key_iv: "3e6f7dd07f7ed8a71562fa875b15f314"
server_key_version: "encryptionkey@0"
I20260927 07:04:21.060154 29859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:04:21.079005 29859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:04:21.083648 29859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:04:21.083928 29859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:04:21.084964 29859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:04:21.086385 29859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:04:21.086493 29859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:21.086625 29859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:04:21.086701 29859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:21.118049 29859 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:44001
I20260927 07:04:21.118065 30006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:44001 every 8 connection(s)
I20260927 07:04:21.119642 29859 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:04:21.130044 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 29859
I20260927 07:04:21.130810 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:42677
--local_ip_for_outbound_sockets=127.13.121.194
--tserver_master_addrs=127.13.121.254:38269
--webserver_port=42581
--webserver_interface=127.13.121.194
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:04:21 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492661, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:04:21.146162 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:21.122585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:45687 (local address 127.13.121.254:38269)
0927 07:04:21.123239 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:21.123251 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:21.123746 (+ 495us) server_negotiation.cc:407] Connection header received
0927 07:04:21.126038 (+ 2292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:21.126054 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:21.126515 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:21.126877 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:21.128749 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:21.130777 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:21.132673 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:21.133710 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:21.133925 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:21.137929 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:21.137939 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:21.137956 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:21.137999 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:21.141184 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:21.142125 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:21.142134 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:21.142140 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:21.142346 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:21.142816 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:21.142820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:21.142822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:21.143427 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:21.143634 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:21.144469 (+ 835us) server_negotiation.cc:299] Negotiation successful
0927 07:04:21.144886 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":256,"threads_started":1}
I20260927 07:04:21.146162 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:21.122657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:45687)
0927 07:04:21.123508 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:21.123556 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:21.125364 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:21.127226 (+ 1862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:21.127235 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:21.127871 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:04:21.128490 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:21.128512 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:21.131062 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:21.131068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:21.132475 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:21.132494 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:21.134018 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:21.134024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:21.134253 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:21.134963 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:04:21.134988 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:04:21.137569 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:04:21.141420 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:21.141430 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:21.141443 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:21.141951 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:21.142476 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:21.142479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:21.142482 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:21.142690 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:21.143579 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:04:21.143588 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:04:21.143948 (+ 360us) client_negotiation.cc:769] Sending connection context
0927 07:04:21.144195 (+ 247us) client_negotiation.cc:240] Negotiation successful
0927 07:04:21.144806 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":328,"threads_started":1}
I20260927 07:04:21.149173 30007 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38269
I20260927 07:04:21.149552 30007 heartbeater.cc:464] Registering TS with master...
I20260927 07:04:21.150534 30007 heartbeater.cc:511] Master 127.13.121.254:38269 requested a full tablet report, sending...
I20260927 07:04:21.152904 29822 ts_manager.cc:194] Registered new tserver with Master: 4c9c527218ce464da8a12134a73ea6cb (127.13.121.193:44001)
I20260927 07:04:21.154438 29822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:45687
W20260927 07:04:21.462212 30011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:04:21.462510 30011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:04:21.462632 30011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:04:21.470329 30011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:04:21.470464 30011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:04:21.470611 30011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:04:21.481182 30011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:42677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=42581
--tserver_master_addrs=127.13.121.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:04:21.482631 30011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:04:21.484004 30011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:04:21.497190 30016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:21.497249 30017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:21.497170 30019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:04:21.497762 30011 server_base.cc:1034] running on GCE node
I20260927 07:04:21.498417 30011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:04:21.499538 30011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:04:21.500756 30011 hybrid_clock.cc:648] HybridClock initialized: now 1790492661500703 us; error 44 us; skew 500 ppm
I20260927 07:04:21.501082 30011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:04:21 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492661, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:04:21.505872 30011 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:04:21.508213 30011 webserver.cc:460] Webserver started at http://127.13.121.194:42581/ using document root <none> and password file <none>
I20260927 07:04:21.509094 30011 fs_manager.cc:359] Metadata directory not provided
I20260927 07:04:21.509227 30011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:04:21.517233 30011 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.003s
I20260927 07:04:21.521564 30026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:21.523573 30011 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 07:04:21.523784 30011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "947c8106699644f0b5058c2ea438aa2c"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1a083542fe069452ade0a5a9d3f27a84"
server_key_iv: "72596f6cfed49f81b4282ea2655fb61f"
server_key_version: "encryptionkey@0"
I20260927 07:04:21.525244 30011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:04:21.545192 30011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:04:21.549533 30011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:04:21.549777 30011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:04:21.550753 30011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:04:21.552184 30011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:04:21.552282 30011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:21.552366 30011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:04:21.552423 30011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:21.583374 30011 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:42677
I20260927 07:04:21.583479 30138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:42677 every 8 connection(s)
I20260927 07:04:21.584949 30011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:04:21.594544 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 30011
I20260927 07:04:21.595177 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:45665
--local_ip_for_outbound_sockets=127.13.121.195
--tserver_master_addrs=127.13.121.254:38269
--webserver_port=33633
--webserver_interface=127.13.121.195
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 07:04:21 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492661, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:04:21.606331 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:21.587576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:38457 (local address 127.13.121.254:38269)
0927 07:04:21.587759 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:21.587764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:21.588994 (+ 1230us) server_negotiation.cc:407] Connection header received
0927 07:04:21.590928 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:21.590933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:21.591012 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:21.591101 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:21.592720 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:21.593455 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:21.595226 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:21.595820 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:21.595957 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:21.599746 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:21.599753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:21.599758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:21.599809 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:21.602880 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:21.603906 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:21.603911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:21.603913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:21.604038 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:21.604513 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:21.604518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:21.604521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:21.604901 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:21.605016 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:21.605890 (+ 874us) server_negotiation.cc:299] Negotiation successful
0927 07:04:21.606084 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 07:04:21.607751 30141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:21.588065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:38457)
0927 07:04:21.588826 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:21.588866 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:21.590600 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:21.591370 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:21.591379 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:21.591937 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:04:21.592533 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:21.592551 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:21.593632 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:21.593636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:21.595024 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:21.595038 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:21.596047 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:21.596053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:21.596284 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:21.596983 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:04:21.597006 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:04:21.599452 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:04:21.603058 (+ 3606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:21.603069 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:21.603090 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:21.603788 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:21.604132 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:21.604136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:21.604139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:21.604405 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:21.605019 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:04:21.605028 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:04:21.605493 (+ 465us) client_negotiation.cc:769] Sending connection context
0927 07:04:21.605783 (+ 290us) client_negotiation.cc:240] Negotiation successful
0927 07:04:21.606334 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":291,"threads_started":1}
I20260927 07:04:21.609315 30139 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38269
I20260927 07:04:21.609647 30139 heartbeater.cc:464] Registering TS with master...
I20260927 07:04:21.610504 30139 heartbeater.cc:511] Master 127.13.121.254:38269 requested a full tablet report, sending...
I20260927 07:04:21.612138 29822 ts_manager.cc:194] Registered new tserver with Master: 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677)
I20260927 07:04:21.612967 29822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:38457
W20260927 07:04:21.928843 30142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:04:21.929160 30142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:04:21.929296 30142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:04:21.936843 30142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:04:21.936992 30142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:04:21.937155 30142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:04:21.947300 30142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:45665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=33633
--tserver_master_addrs=127.13.121.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:04:21.948601 30142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:04:21.950011 30142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:04:21.963202 30147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:21.963171 30148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:21.963182 30150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:04:21.963581 30142 server_base.cc:1034] running on GCE node
I20260927 07:04:21.964397 30142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:04:21.965355 30142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:04:21.966548 30142 hybrid_clock.cc:648] HybridClock initialized: now 1790492661966503 us; error 36 us; skew 500 ppm
I20260927 07:04:21.966851 30142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:04:21 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492661, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:04:21.969897 30142 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:04:21.971424 30142 webserver.cc:460] Webserver started at http://127.13.121.195:33633/ using document root <none> and password file <none>
I20260927 07:04:21.972167 30142 fs_manager.cc:359] Metadata directory not provided
I20260927 07:04:21.972304 30142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:04:21.978024 30142 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260927 07:04:21.981042 30157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:21.982486 30142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260927 07:04:21.982662 30142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "cfda194b51194355b9133bd3263bd255"
format_stamp: "Formatted at 2026-09-27 07:03:59 on dist-test-slave-p6hz"
server_key: "c5aefe99c67f8c3d50e589524d6ef759"
server_key_iv: "252438f659912577561e8b0f4e391b1f"
server_key_version: "encryptionkey@0"
I20260927 07:04:21.983779 30142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:04:22.007414 30142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:04:22.011786 30142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:04:22.012050 30142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:04:22.013104 30142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:04:22.014614 30142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:04:22.014705 30142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:22.014816 30142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:04:22.014881 30142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:22.046762 30142 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:45665
I20260927 07:04:22.046819 30269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:45665 every 8 connection(s)
I20260927 07:04:22.048266 30142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:04:22.051944 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 30142
Sep 27 07:04:22 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492661, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:04:22.069267 30273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.052859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43778 (local address 127.13.121.254:38269)
0927 07:04:22.053322 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.053328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.055018 (+ 1690us) server_negotiation.cc:407] Connection header received
0927 07:04:22.055219 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.055223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.055284 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.055372 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:22.057240 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.058149 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.059748 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.060273 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.060408 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.062991 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:22.062999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:22.063003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:22.063039 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:22.066538 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.067295 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.067301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.067304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.067434 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.067790 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.067794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.067796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.068090 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:22.068336 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.068680 (+ 344us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.068901 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":232,"threads_started":1}
I20260927 07:04:22.070255 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.050856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:48291 (local address 127.13.121.254:38269)
0927 07:04:22.051068 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.051072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.052432 (+ 1360us) server_negotiation.cc:407] Connection header received
0927 07:04:22.054470 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.054476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.054587 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.054752 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:22.056885 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.057666 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.059468 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.060082 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.060190 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.063938 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:22.063949 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:22.063954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:22.064011 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:22.067035 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.067809 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.067815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.067818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.067965 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.068395 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.068400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.068404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.068652 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:22.068776 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.069608 (+ 832us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.069910 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 07:04:22.071571 30272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.051370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:48291)
0927 07:04:22.052255 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.052311 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:22.054170 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:22.055026 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:22.055039 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:22.055813 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:04:22.056675 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.056706 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.057835 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.057842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.059313 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.059328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.060373 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.060382 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.060686 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.061366 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:04:22.061396 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:04:22.063677 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:04:22.067203 (+ 3526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:22.067213 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:22.067230 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:22.067683 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:22.068074 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:22.068079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:22.068083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:22.068279 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:22.068777 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:04:22.068786 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:04:22.069198 (+ 412us) client_negotiation.cc:769] Sending connection context
0927 07:04:22.069515 (+ 317us) client_negotiation.cc:240] Negotiation successful
0927 07:04:22.070096 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":323,"threads_started":1}
I20260927 07:04:22.073284 30270 heartbeater.cc:347] Connected to a master server at 127.13.121.254:38269
I20260927 07:04:22.073617 30270 heartbeater.cc:464] Registering TS with master...
I20260927 07:04:22.074493 30270 heartbeater.cc:511] Master 127.13.121.254:38269 requested a full tablet report, sending...
I20260927 07:04:22.076004 29822 ts_manager.cc:194] Registered new tserver with Master: cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665)
I20260927 07:04:22.076858 29822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:48291
I20260927 07:04:22.084903 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:04:22.091924 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.085796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.13.121.254:38269)
0927 07:04:22.085969 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.085989 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.086420 (+ 431us) server_negotiation.cc:407] Connection header received
0927 07:04:22.086667 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.086673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.086864 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.087000 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:04:22.087509 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.088368 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.089945 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.090360 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.090470 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.091009 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:04:22.091262 (+ 253us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0927 07:04:22.091657 (+ 395us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43780: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260927 07:04:22.092057 30009 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43780: token expired
W20260927 07:04:22.092645 29741 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.121.254:38269)
I20260927 07:04:22.107956 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.093175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43796 (local address 127.13.121.254:38269)
0927 07:04:22.093332 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.093336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.093354 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:04:22.093480 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.093484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.093545 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.093623 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:22.095119 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.096008 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.098154 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.098869 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.099014 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.102691 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:22.102698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:22.102701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:22.102744 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:22.105328 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.106137 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.106142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.106144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.106275 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.106591 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.106594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.106596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.106932 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:22.107124 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.107519 (+ 395us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.107711 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:04:22.111744 29822 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:04:22.119809 29822 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:04:22.147046 30280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.133014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42677 (local address 127.0.0.1:37018)
0927 07:04:22.133930 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.133981 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:22.134154 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:22.135204 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:22.135212 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:22.135239 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:04:22.135578 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.135592 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.139377 (+ 3785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.139384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.142150 (+ 2766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.142177 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.146187 (+ 4010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.146194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.146429 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.146482 (+ 53us) client_negotiation.cc:769] Sending connection context
0927 07:04:22.146582 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 07:04:22.146732 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":497,"threads_started":1}
I20260927 07:04:22.148186 30281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.133202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.13.121.194:42677)
0927 07:04:22.134526 (+ 1324us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.134532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.134559 (+ 27us) server_negotiation.cc:407] Connection header received
0927 07:04:22.134663 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.134671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.134950 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.135785 (+ 835us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:04:22.136045 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.139173 (+ 3128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.144093 (+ 4920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.145881 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.147389 (+ 1508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.147596 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.147682 (+ 86us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.147868 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1199,"thread_start_us":1066,"threads_started":1}
I20260927 07:04:22.152320 30074 tablet_service.cc:1467] Processing CreateTablet for tablet 77aeda21cb8d43b693530db7a0dc4c2f (DEFAULT_TABLE table=test-table [id=348cb3aff57b406f84fb6e33ac9121de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:04:22.154237 30074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77aeda21cb8d43b693530db7a0dc4c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:04:22.156543 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.135048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:45665 (local address 127.0.0.1:34762)
0927 07:04:22.145420 (+ 10372us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.145445 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:22.145603 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:22.147569 (+ 1966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:22.147575 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:22.147610 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:04:22.148037 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.148050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.151587 (+ 3537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.151593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.153944 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.153967 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.155716 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.155724 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.155890 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.155926 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 07:04:22.156027 (+ 101us) client_negotiation.cc:240] Negotiation successful
0927 07:04:22.156180 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10178,"thread_start_us":10658,"threads_started":1}
I20260927 07:04:22.156603 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.135616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.13.121.195:45665)
0927 07:04:22.146501 (+ 10885us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.146512 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.146555 (+ 43us) server_negotiation.cc:407] Connection header received
0927 07:04:22.146673 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.146684 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.146998 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.148284 (+ 1286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:04:22.148590 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.151427 (+ 2837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.154134 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.155482 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.155694 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.155901 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.156102 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.156281 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10672,"thread_start_us":10514,"threads_started":1}
I20260927 07:04:22.158267 30007 heartbeater.cc:503] Master 127.13.121.254:38269 was elected leader, sending a full tablet report...
I20260927 07:04:22.161352 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.134416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:44001 (local address 127.0.0.1:60198)
0927 07:04:22.134962 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.134985 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:22.135101 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:22.150145 (+ 15044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:22.150150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:22.150219 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:04:22.150519 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.150531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.152236 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.152241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.154529 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.155252 (+ 723us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.160432 (+ 5180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.160440 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.160664 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.160702 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 07:04:22.160818 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 07:04:22.160983 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":277,"threads_started":1}
I20260927 07:04:22.161644 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.134931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.13.121.193:44001)
0927 07:04:22.149614 (+ 14683us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.149621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.149645 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:04:22.149730 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.149736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.149957 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.150140 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:04:22.150705 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.152074 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.155464 (+ 3390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.158513 (+ 3049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.160039 (+ 1526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.160252 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.160897 (+ 645us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.161095 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14539,"thread_start_us":14401,"threads_started":1}
I20260927 07:04:22.161010 30205 tablet_service.cc:1467] Processing CreateTablet for tablet 77aeda21cb8d43b693530db7a0dc4c2f (DEFAULT_TABLE table=test-table [id=348cb3aff57b406f84fb6e33ac9121de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:04:22.162719 30205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77aeda21cb8d43b693530db7a0dc4c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:04:22.165899 29942 tablet_service.cc:1467] Processing CreateTablet for tablet 77aeda21cb8d43b693530db7a0dc4c2f (DEFAULT_TABLE table=test-table [id=348cb3aff57b406f84fb6e33ac9121de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:04:22.167518 29942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77aeda21cb8d43b693530db7a0dc4c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:04:22.171223 30286 tablet_bootstrap.cc:492] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: Bootstrap starting.
I20260927 07:04:22.175236 30286 tablet_bootstrap.cc:654] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: Neither blocks nor log segments found. Creating new log.
I20260927 07:04:22.176684 30286 log.cc:824] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:04:22.177425 30287 tablet_bootstrap.cc:492] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: Bootstrap starting.
I20260927 07:04:22.181237 30287 tablet_bootstrap.cc:654] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: Neither blocks nor log segments found. Creating new log.
I20260927 07:04:22.181603 30286 tablet_bootstrap.cc:492] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: No bootstrap required, opened a new log
I20260927 07:04:22.181877 30286 ts_tablet_manager.cc:1397] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260927 07:04:22.182631 30287 log.cc:824] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: Log is configured to *not* fsync() on all Append() calls
I20260927 07:04:22.183399 30289 tablet_bootstrap.cc:492] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Bootstrap starting.
I20260927 07:04:22.187582 30289 tablet_bootstrap.cc:654] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Neither blocks nor log segments found. Creating new log.
I20260927 07:04:22.189337 30289 log.cc:824] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:04:22.189872 30287 tablet_bootstrap.cc:492] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: No bootstrap required, opened a new log
I20260927 07:04:22.190194 30287 ts_tablet_manager.cc:1397] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260927 07:04:22.193239 30289 tablet_bootstrap.cc:492] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: No bootstrap required, opened a new log
I20260927 07:04:22.192888 30286 raft_consensus.cc:348] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:22.193408 30286 raft_consensus.cc:374] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:04:22.193902 30289 ts_tablet_manager.cc:1397] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260927 07:04:22.193881 30286 raft_consensus.cc:729] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947c8106699644f0b5058c2ea438aa2c, State: Initialized, Role: FOLLOWER
I20260927 07:04:22.194799 30286 consensus_queue.cc:260] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:22.196224 30139 heartbeater.cc:503] Master 127.13.121.254:38269 was elected leader, sending a full tablet report...
I20260927 07:04:22.197196 30286 ts_tablet_manager.cc:1428] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260927 07:04:22.200249 30287 raft_consensus.cc:348] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:22.201071 30287 raft_consensus.cc:374] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:04:22.201189 30287 raft_consensus.cc:729] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfda194b51194355b9133bd3263bd255, State: Initialized, Role: FOLLOWER
I20260927 07:04:22.202005 30287 consensus_queue.cc:260] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:22.203299 30270 heartbeater.cc:503] Master 127.13.121.254:38269 was elected leader, sending a full tablet report...
I20260927 07:04:22.203840 30287 ts_tablet_manager.cc:1428] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260927 07:04:22.204886 30289 raft_consensus.cc:348] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:22.205324 30289 raft_consensus.cc:374] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:04:22.205435 30289 raft_consensus.cc:729] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9c527218ce464da8a12134a73ea6cb, State: Initialized, Role: FOLLOWER
I20260927 07:04:22.206122 30289 consensus_queue.cc:260] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:22.207999 30289 ts_tablet_manager.cc:1428] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260927 07:04:22.292703 30294 raft_consensus.cc:482] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:04:22.292999 30294 raft_consensus.cc:504] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:22.294655 30294 leader_election.cc:288] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677), cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665)
W20260927 07:04:22.299754 30271 tablet.cc:2369] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:04:22.302496 30281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.295059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:56653 (local address 127.13.121.194:42677)
0927 07:04:22.295293 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.295299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.295896 (+ 597us) server_negotiation.cc:407] Connection header received
0927 07:04:22.296084 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.296089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.296149 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.296223 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:04:22.296910 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.297847 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.300431 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.301622 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.301791 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.301870 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.302169 (+ 299us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.302278 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:04:22.302742 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.295071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42677 (local address 127.13.121.193:56653)
0927 07:04:22.295754 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.295773 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:22.295930 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:22.296251 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:22.296255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:22.296281 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:04:22.296740 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.296755 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.298012 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.298017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.300180 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.300197 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.301856 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.301861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.301991 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.302019 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 07:04:22.302139 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 07:04:22.302282 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":21,"thread_start_us":403,"threads_started":1}
I20260927 07:04:22.302846 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.296198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:50087 (local address 127.13.121.195:45665)
0927 07:04:22.296400 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.296406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.296965 (+ 559us) server_negotiation.cc:407] Connection header received
0927 07:04:22.297049 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.297054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.297132 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.297234 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:04:22.297628 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.298543 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.300782 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.301964 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.302078 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.302139 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.302492 (+ 353us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.302621 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 07:04:22.302992 30296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.295893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:45665 (local address 127.13.121.193:50087)
0927 07:04:22.296819 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.296841 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:22.296967 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:22.297234 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:22.297237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:22.297251 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:04:22.297501 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.297508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.298665 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.298670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.300647 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:22.300658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.302172 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.302177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:22.302331 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.302362 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 07:04:22.302461 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 07:04:22.302588 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"mutex_wait_us":1,"thread_start_us":586,"threads_started":1}
I20260927 07:04:22.303910 30094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77aeda21cb8d43b693530db7a0dc4c2f" candidate_uuid: "4c9c527218ce464da8a12134a73ea6cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947c8106699644f0b5058c2ea438aa2c" is_pre_election: true
I20260927 07:04:22.303910 30225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77aeda21cb8d43b693530db7a0dc4c2f" candidate_uuid: "4c9c527218ce464da8a12134a73ea6cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfda194b51194355b9133bd3263bd255" is_pre_election: true
I20260927 07:04:22.304454 30225 raft_consensus.cc:2454] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9c527218ce464da8a12134a73ea6cb in term 0.
I20260927 07:04:22.304454 30094 raft_consensus.cc:2454] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9c527218ce464da8a12134a73ea6cb in term 0.
I20260927 07:04:22.305150 29896 leader_election.cc:302] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9c527218ce464da8a12134a73ea6cb, 947c8106699644f0b5058c2ea438aa2c; no voters:
I20260927 07:04:22.305552 30294 raft_consensus.cc:2781] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:04:22.305683 30294 raft_consensus.cc:482] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:04:22.305783 30294 raft_consensus.cc:3037] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:04:22.307987 30294 raft_consensus.cc:504] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:22.308568 30294 leader_election.cc:288] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 1 election: Requested vote from peers 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677), cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665)
I20260927 07:04:22.309139 30094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77aeda21cb8d43b693530db7a0dc4c2f" candidate_uuid: "4c9c527218ce464da8a12134a73ea6cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947c8106699644f0b5058c2ea438aa2c"
I20260927 07:04:22.309163 30225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77aeda21cb8d43b693530db7a0dc4c2f" candidate_uuid: "4c9c527218ce464da8a12134a73ea6cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfda194b51194355b9133bd3263bd255"
I20260927 07:04:22.309369 30094 raft_consensus.cc:3037] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:04:22.309361 30225 raft_consensus.cc:3037] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:04:22.312513 30225 raft_consensus.cc:2454] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9c527218ce464da8a12134a73ea6cb in term 1.
I20260927 07:04:22.312513 30094 raft_consensus.cc:2454] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9c527218ce464da8a12134a73ea6cb in term 1.
I20260927 07:04:22.312968 29896 leader_election.cc:302] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9c527218ce464da8a12134a73ea6cb, 947c8106699644f0b5058c2ea438aa2c; no voters:
I20260927 07:04:22.313254 30294 raft_consensus.cc:2781] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:04:22.313746 30294 raft_consensus.cc:686] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 LEADER]: Becoming Leader. State: Replica: 4c9c527218ce464da8a12134a73ea6cb, State: Running, Role: LEADER
I20260927 07:04:22.314200 30294 consensus_queue.cc:237] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:22.318542 29822 catalog_manager.cc:5495] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c9c527218ce464da8a12134a73ea6cb (127.13.121.193). New cstate: current_term: 1 leader_uuid: "4c9c527218ce464da8a12134a73ea6cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } health_report { overall_health: UNKNOWN } } }
W20260927 07:04:22.336987 30140 tablet.cc:2369] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:04:22.344241 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.338727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.13.121.193:44001)
0927 07:04:22.338947 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.338952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.338970 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:04:22.339072 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.339075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.339213 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.339353 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:04:22.339767 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.340522 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.341974 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.342399 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.342499 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.343052 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:04:22.343674 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:04:22.343790 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.343915 (+ 125us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.343987 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260927 07:04:22.353097 30225 raft_consensus.cc:1261] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [term 1 FOLLOWER]: Refusing update from remote peer 4c9c527218ce464da8a12134a73ea6cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:04:22.353097 30094 raft_consensus.cc:1261] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [term 1 FOLLOWER]: Refusing update from remote peer 4c9c527218ce464da8a12134a73ea6cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:04:22.354198 30297 consensus_queue.cc:1035] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:04:22.354578 30294 consensus_queue.cc:1035] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:04:22.366400 30299 mvcc.cc:205] Tried to move back new op lower bound from 7333857944987082752 to 7333857944842829824. Current Snapshot: MvccSnapshot[applied={T|T < 7333857944987082752}]
I20260927 07:04:22.367276 30301 mvcc.cc:205] Tried to move back new op lower bound from 7333857944987082752 to 7333857944842829824. Current Snapshot: MvccSnapshot[applied={T|T < 7333857944987082752}]
I20260927 07:04:22.382943 29821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:04:22.383278 29821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:04:22.387416 29821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Sending DeleteTablet for 3 replicas of tablet 77aeda21cb8d43b693530db7a0dc4c2f
I20260927 07:04:22.388764 30074 tablet_service.cc:1514] Processing DeleteTablet for tablet 77aeda21cb8d43b693530db7a0dc4c2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:04:22 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:37018
I20260927 07:04:22.388924 29942 tablet_service.cc:1514] Processing DeleteTablet for tablet 77aeda21cb8d43b693530db7a0dc4c2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:04:22 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:60198
I20260927 07:04:22.389194 30205 tablet_service.cc:1514] Processing DeleteTablet for tablet 77aeda21cb8d43b693530db7a0dc4c2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:04:22 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:34762
I20260927 07:04:22.389729 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 29859
I20260927 07:04:22.402122 30310 tablet_replica.cc:330] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: stopping tablet replica
I20260927 07:04:22.402839 30310 raft_consensus.cc:2229] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:04:22.403604 30310 raft_consensus.cc:2258] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:04:22.406178 30310 ts_tablet_manager.cc:1905] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:04:22.413729 30310 ts_tablet_manager.cc:1918] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:04:22.414036 30310 log.cc:1196] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/77aeda21cb8d43b693530db7a0dc4c2f
I20260927 07:04:22.414714 30310 ts_tablet_manager.cc:1939] T 77aeda21cb8d43b693530db7a0dc4c2f P cfda194b51194355b9133bd3263bd255: Deleting consensus metadata
I20260927 07:04:22.416954 29807 catalog_manager.cc:4841] TS cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665): tablet 77aeda21cb8d43b693530db7a0dc4c2f (table test-table [id=348cb3aff57b406f84fb6e33ac9121de]) successfully deleted
I20260927 07:04:22.419312 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.390889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.13.121.193:44001)
0927 07:04:22.391078 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.391084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.391109 (+ 25us) server_negotiation.cc:407] Connection header received
0927 07:04:22.391197 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.391203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.391301 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.391419 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:22.393789 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.395211 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.397513 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.398334 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.398483 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.407119 (+ 8636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:22.407162 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:22.407184 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:22.407255 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:22.411922 (+ 4667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.413325 (+ 1403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.413337 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.413341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.413597 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.414933 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.414939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.415026 (+ 87us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.415676 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:22.416064 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.418497 (+ 2433us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.418828 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 07:04:22.420082 30002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:04:22.421502 30308 tablet_replica.cc:330] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: stopping tablet replica
I20260927 07:04:22.421979 30308 raft_consensus.cc:2229] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:04:22.422482 30308 raft_consensus.cc:2258] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:04:22.424707 30308 ts_tablet_manager.cc:1905] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:04:22.429801 30308 ts_tablet_manager.cc:1918] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:04:22.429952 30308 log.cc:1196] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/77aeda21cb8d43b693530db7a0dc4c2f
I20260927 07:04:22.430384 30308 ts_tablet_manager.cc:1939] T 77aeda21cb8d43b693530db7a0dc4c2f P 947c8106699644f0b5058c2ea438aa2c: Deleting consensus metadata
I20260927 07:04:22.432060 29807 catalog_manager.cc:4841] TS 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677): tablet 77aeda21cb8d43b693530db7a0dc4c2f (table test-table [id=348cb3aff57b406f84fb6e33ac9121de]) successfully deleted
I20260927 07:04:22.520831 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 29859
I20260927 07:04:22.521003 30309 tablet_replica.cc:330] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: stopping tablet replica
W20260927 07:04:22.536500 29807 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.121.193:44001 (error 108)
I20260927 07:04:22.536677 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 30011
I20260927 07:04:22.538282 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.537847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:44001 (local address 127.0.0.1:60220)
0927 07:04:22.538012 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.538102 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260927 07:04:22.538580 29807 catalog_manager.cc:4584] TS 4c9c527218ce464da8a12134a73ea6cb (127.13.121.193:44001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77aeda21cb8d43b693530db7a0dc4c2f: Network error: Client connection negotiation failed: client connection to 127.13.121.193:44001: connect: Connection refused (error 111)
I20260927 07:04:22.550494 30281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.537592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.13.121.194:42677)
0927 07:04:22.537764 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.537770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.537789 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:04:22.537992 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.537999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.538069 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.538164 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:22.539609 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.540388 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.541647 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.542041 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.542138 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.544842 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:22.544858 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:22.544875 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:22.544922 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:22.547766 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.548535 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.548542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.548544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.548669 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.549042 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.549045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.549047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.549395 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:22.549614 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.549969 (+ 355us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.550213 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 07:04:22.551287 30134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:04:22.592427 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.591901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:44001 (local address 127.0.0.1:60234)
0927 07:04:22.592120 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.592251 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260927 07:04:22.593425 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.593054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:44001 (local address 127.0.0.1:60236)
0927 07:04:22.593189 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.593275 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 07:04:22.640023 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 30011
I20260927 07:04:22.653870 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.653418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:44001 (local address 127.0.0.1:60252)
0927 07:04:22.653612 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.653702 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260927 07:04:22.654709 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 30142
I20260927 07:04:22.655089 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.654687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:44001 (local address 127.0.0.1:60258)
0927 07:04:22.654857 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.654954 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 07:04:22.667838 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.655469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.13.121.195:45665)
0927 07:04:22.655656 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:22.655662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:22.655683 (+ 21us) server_negotiation.cc:407] Connection header received
0927 07:04:22.655866 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:22.655870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:22.655951 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:22.656035 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:22.657372 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.658107 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.659471 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:22.659868 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:22.659981 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:22.662398 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:22.662414 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:22.662430 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:22.662465 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:22.665105 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.665798 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.665806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.665808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.665952 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:22.666292 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:22.666296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:22.666298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:22.666660 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:22.666949 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:22.667312 (+ 363us) server_negotiation.cc:299] Negotiation successful
0927 07:04:22.667546 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 07:04:22.668516 30265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:04:22.735656 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.735173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:44001 (local address 127.0.0.1:60272)
0927 07:04:22.735389 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.735484 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260927 07:04:22.736682 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:22.736290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:44001 (local address 127.0.0.1:60282)
0927 07:04:22.736494 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:22.736570 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260927 07:04:22.757817 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 30142
I20260927 07:04:22.771322 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 29789
I20260927 07:04:22.772552 29852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:04:22.889452 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 29789
I20260927 07:04:42.908378 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38269
--webserver_interface=127.13.121.254
--webserver_port=38569
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:04:43.243873 30316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:04:43.244174 30316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:04:43.244272 30316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:04:43.251573 30316 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 07:04:43.251677 30316 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 07:04:43.251724 30316 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 07:04:43.251771 30316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:04:43.251824 30316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:04:43.251895 30316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:04:43.251941 30316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:04:43.251992 30316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:04:43.262147 30316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=38569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:04:43.263542 30316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:04:43.264860 30316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:04:43.277150 30322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:43.277124 30324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:43.277613 30321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:04:43.277609 30316 server_base.cc:1034] running on GCE node
I20260927 07:04:43.278518 30316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:04:43.279922 30316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:04:43.281129 30316 hybrid_clock.cc:648] HybridClock initialized: now 1790492683281072 us; error 38 us; skew 500 ppm
I20260927 07:04:43.281446 30316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:04:43 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:04:43.284539 30316 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:04:43.285912 30316 webserver.cc:460] Webserver started at http://127.13.121.254:38569/ using document root <none> and password file <none>
I20260927 07:04:43.286544 30316 fs_manager.cc:359] Metadata directory not provided
I20260927 07:04:43.286625 30316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:04:43.292273 30316 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 07:04:43.295084 30331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:43.296499 30316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:04:43.296645 30316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "63e3f2e2e5b14940bcefa3b7530226ea"
format_stamp: "Formatted at 2026-09-27 07:03:57 on dist-test-slave-p6hz"
server_key: "ce46c426a280bec4843cc4afe652e47c"
server_key_iv: "6aca60c991aafffc3a2decbcb2412509"
server_key_version: "encryptionkey@0"
I20260927 07:04:43.297544 30316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:04:43.328814 30316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:04:43.333153 30316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:04:43.333411 30316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:04:43.350970 30316 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:38269
I20260927 07:04:43.350999 30382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:38269 every 8 connection(s)
I20260927 07:04:43.352561 30316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:04:43.359488 30383 sys_catalog.cc:263] Verifying existing consensus state
I20260927 07:04:43.362288 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 30316
I20260927 07:04:43.363178 30383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Bootstrap starting.
I20260927 07:04:43.363395 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:44001
--local_ip_for_outbound_sockets=127.13.121.193
--tserver_master_addrs=127.13.121.254:38269
--webserver_port=45579
--webserver_interface=127.13.121.193
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:04:43.385691 30383 log.cc:824] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Log is configured to *not* fsync() on all Append() calls
I20260927 07:04:43.403209 30383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:04:43.403775 30383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Bootstrap complete.
I20260927 07:04:43.412786 30383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:04:43.413736 30383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e3f2e2e5b14940bcefa3b7530226ea, State: Initialized, Role: FOLLOWER
I20260927 07:04:43.414376 30383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:04:43.414597 30383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:04:43.414708 30383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:04:43.414867 30383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 2 FOLLOWER]: Advancing to term 3
I20260927 07:04:43.418846 30383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:04:43.419389 30383 leader_election.cc:302] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63e3f2e2e5b14940bcefa3b7530226ea; no voters:
I20260927 07:04:43.420030 30383 leader_election.cc:288] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 07:04:43.420177 30387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 3 FOLLOWER]: Leader election won for term 3
I20260927 07:04:43.421471 30387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [term 3 LEADER]: Becoming Leader. State: Replica: 63e3f2e2e5b14940bcefa3b7530226ea, State: Running, Role: LEADER
I20260927 07:04:43.422065 30387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } }
I20260927 07:04:43.422451 30383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:04:43.424710 30389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63e3f2e2e5b14940bcefa3b7530226ea. Latest consensus state: current_term: 3 leader_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } } }
I20260927 07:04:43.424788 30388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e3f2e2e5b14940bcefa3b7530226ea" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38269 } } }
I20260927 07:04:43.425168 30389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: This master's current role is: LEADER
I20260927 07:04:43.425174 30388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea [sys.catalog]: This master's current role is: LEADER
I20260927 07:04:43.426045 30392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:04:43.448702 30392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:04:43.454689 30392 catalog_manager.cc:1259] Loaded cluster ID: f9d6933dde764e1c89649558d4a09369
I20260927 07:04:43.454932 30392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:04:43.459482 30392 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:04:43.461419 30392 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Loaded TSK: 0
I20260927 07:04:43.461694 30392 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 63e3f2e2e5b14940bcefa3b7530226ea: Loaded TSK: 1
F20260927 07:04:43.462882 30392 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff7fc3db9ad google::LogMessage::Fail() at ??:0
@ 0x7ff7fc3de88a google::LogMessage::SendToLog() at ??:0
@ 0x7ff7fc3db4de google::LogMessage::Flush() at ??:0
@ 0x7ff7fc3dca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff80013f7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff80012e73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff811e0a1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff811e09783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff811e952a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7ff811e950c4 std::_Function_handler<>::_M_invoke() at ??:0
I20260927 07:04:43.495676 30405 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7ff811edf5e9 std::function<>::operator()() at ??:0
@ 0x7ff811e0dcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7ff811e0bc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff811e93610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7ff811e93439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff7ff027345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff7ff02a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7ff7ff02a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff7ff005eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff8022df6db start_thread at ??:0
@ 0x7ff7fb27561f clone at ??:0
W20260927 07:04:43.734163 30385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:04:43.734468 30385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:04:43.734597 30385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:04:43.742022 30385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:04:43.742154 30385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:04:43.742308 30385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:04:43.752235 30385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:44001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=45579
--tserver_master_addrs=127.13.121.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:04:43.753548 30385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:04:43.754835 30385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:04:43.767988 30433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:43.768042 30430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:43.768021 30431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:04:43.768772 30385 server_base.cc:1034] running on GCE node
I20260927 07:04:43.769398 30385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:04:43.770390 30385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:04:43.771595 30385 hybrid_clock.cc:648] HybridClock initialized: now 1790492683771544 us; error 44 us; skew 500 ppm
I20260927 07:04:43.771922 30385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:04:43 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:04:43.774920 30385 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:04:43.776458 30385 webserver.cc:460] Webserver started at http://127.13.121.193:45579/ using document root <none> and password file <none>
I20260927 07:04:43.777149 30385 fs_manager.cc:359] Metadata directory not provided
I20260927 07:04:43.777271 30385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:04:43.782698 30385 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:04:43.785660 30440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:43.787087 30385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:04:43.787319 30385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "4c9c527218ce464da8a12134a73ea6cb"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1f5377e3840bd51730c5286fecc43abd"
server_key_iv: "3e6f7dd07f7ed8a71562fa875b15f314"
server_key_version: "encryptionkey@0"
I20260927 07:04:43.788326 30385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:04:43.805044 30385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:04:43.809500 30385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:04:43.809762 30385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:04:43.810745 30385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:04:43.812587 30447 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260927 07:04:43.818352 30385 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 07:04:43.818498 30385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260927 07:04:43.818632 30385 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 07:04:43.821928 30385 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 07:04:43.822019 30385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.001s
I20260927 07:04:43.822373 30447 tablet_bootstrap.cc:492] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Bootstrap starting.
I20260927 07:04:43.848496 30447 log.cc:824] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:04:43.857476 30385 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:44001
I20260927 07:04:43.857509 30554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:44001 every 8 connection(s)
I20260927 07:04:43.858775 30385 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:04:43.861287 30447 tablet_bootstrap.cc:492] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 07:04:43.861368 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 30385
I20260927 07:04:43.861914 30447 tablet_bootstrap.cc:492] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Bootstrap complete.
I20260927 07:04:43.862006 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:42677
--local_ip_for_outbound_sockets=127.13.121.194
--tserver_master_addrs=127.13.121.254:38269
--webserver_port=42581
--webserver_interface=127.13.121.194
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:04:43.863461 30447 ts_tablet_manager.cc:1397] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Time spent bootstrapping tablet: real 0.041s user 0.033s sys 0.008s
I20260927 07:04:43.864276 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:43.861581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:48995)
0927 07:04:43.862773 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:43.863089 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":889,"thread_start_us":647,"threads_started":1}
W20260927 07:04:43.864929 30555 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:38269 (0 consecutive failures): Network error: Failed to ping master at 127.13.121.254:38269: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
I20260927 07:04:43.865880 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:43.865489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:47297)
0927 07:04:43.865659 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:43.865750 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260927 07:04:43.866935 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:43.866514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:47455)
0927 07:04:43.866675 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:43.866770 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260927 07:04:43.867264 30555 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:04:43.870779 30447 raft_consensus.cc:348] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:43.872042 30447 raft_consensus.cc:729] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9c527218ce464da8a12134a73ea6cb, State: Initialized, Role: FOLLOWER
I20260927 07:04:43.872754 30447 consensus_queue.cc:260] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:43.874408 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:43.874017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:45841)
0927 07:04:43.874175 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:43.874255 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 07:04:43.874415 30447 ts_tablet_manager.cc:1428] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260927 07:04:43.875329 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:43.874972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:37515)
0927 07:04:43.875110 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:43.875209 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 07:04:44.189198 30558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:04:44.189498 30558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:04:44.189628 30558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:04:44.197147 30558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:04:44.197291 30558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:04:44.197443 30558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:04:44.207401 30558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:42677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=42581
--tserver_master_addrs=127.13.121.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:04:44.208666 30558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:04:44.209918 30558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:04:44.222811 30565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:44.222891 30567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:44.222800 30564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:04:44.223872 30558 server_base.cc:1034] running on GCE node
I20260927 07:04:44.224506 30558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:04:44.225478 30558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:04:44.226683 30558 hybrid_clock.cc:648] HybridClock initialized: now 1790492684226627 us; error 43 us; skew 500 ppm
I20260927 07:04:44.226994 30558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:04:44 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:04:44.230029 30558 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:04:44.231498 30558 webserver.cc:460] Webserver started at http://127.13.121.194:42581/ using document root <none> and password file <none>
I20260927 07:04:44.232182 30558 fs_manager.cc:359] Metadata directory not provided
I20260927 07:04:44.232296 30558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:04:44.237587 30558 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260927 07:04:44.240295 30574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:44.241586 30558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 07:04:44.241740 30558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "947c8106699644f0b5058c2ea438aa2c"
format_stamp: "Formatted at 2026-09-27 07:03:58 on dist-test-slave-p6hz"
server_key: "1a083542fe069452ade0a5a9d3f27a84"
server_key_iv: "72596f6cfed49f81b4282ea2655fb61f"
server_key_version: "encryptionkey@0"
I20260927 07:04:44.242713 30558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:04:44.278424 30558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:04:44.282608 30558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:04:44.282858 30558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:04:44.283886 30558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:04:44.285310 30558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:04:44.285411 30558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:44.285514 30558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:04:44.285579 30558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:44.315757 30558 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:42677
I20260927 07:04:44.315835 30686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:42677 every 8 connection(s)
I20260927 07:04:44.317188 30558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:04:44.317715 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 30558
I20260927 07:04:44.318442 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:45665
--local_ip_for_outbound_sockets=127.13.121.195
--tserver_master_addrs=127.13.121.254:38269
--webserver_port=33633
--webserver_interface=127.13.121.195
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:04:44.322450 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:44.320535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:42433)
0927 07:04:44.321335 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:44.321573 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":320,"threads_started":1}
W20260927 07:04:44.323180 30687 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:38269 (0 consecutive failures): Network error: Failed to ping master at 127.13.121.254:38269: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
I20260927 07:04:44.324041 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:44.323723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:54921)
0927 07:04:44.323868 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:44.323936 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 07:04:44.324826 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:44.324518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:54359)
0927 07:04:44.324648 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:44.324723 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 07:04:44.325045 30687 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 07:04:44.644264 30690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:04:44.644554 30690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:04:44.644647 30690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:04:44.652091 30690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:04:44.652211 30690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:04:44.652328 30690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:04:44.662261 30690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:45665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=33633
--tserver_master_addrs=127.13.121.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:04:44.663636 30690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:04:44.664892 30690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:04:44.677578 30695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:44.677616 30696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:04:44.677578 30698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:04:44.678459 30690 server_base.cc:1034] running on GCE node
I20260927 07:04:44.679006 30690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:04:44.679950 30690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:04:44.681147 30690 hybrid_clock.cc:648] HybridClock initialized: now 1790492684681090 us; error 58 us; skew 500 ppm
I20260927 07:04:44.681461 30690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:04:44 dist-test-slave-p6hz krb5kdc[29237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:04:44.684509 30690 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:04:44.685943 30690 webserver.cc:460] Webserver started at http://127.13.121.195:33633/ using document root <none> and password file <none>
I20260927 07:04:44.686599 30690 fs_manager.cc:359] Metadata directory not provided
I20260927 07:04:44.686717 30690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:04:44.692189 30690 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260927 07:04:44.695039 30705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:44.696470 30690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:04:44.696630 30690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "cfda194b51194355b9133bd3263bd255"
format_stamp: "Formatted at 2026-09-27 07:03:59 on dist-test-slave-p6hz"
server_key: "c5aefe99c67f8c3d50e589524d6ef759"
server_key_iv: "252438f659912577561e8b0f4e391b1f"
server_key_version: "encryptionkey@0"
I20260927 07:04:44.697610 30690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:04:44.713855 30690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:04:44.718322 30690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:04:44.718719 30690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:04:44.719940 30690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:04:44.721428 30690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:04:44.721542 30690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:44.721654 30690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:04:44.721728 30690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:04:44.753214 30690 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:45665
I20260927 07:04:44.753279 30817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:45665 every 8 connection(s)
I20260927 07:04:44.754640 30690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:04:44.759585 30820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:44.757685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:39501)
0927 07:04:44.758426 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:44.758668 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":287,"threads_started":1}
W20260927 07:04:44.760252 30818 heartbeater.cc:644] Failed to heartbeat to 127.13.121.254:38269 (0 consecutive failures): Network error: Failed to ping master at 127.13.121.254:38269: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
I20260927 07:04:44.761049 30820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:44.760744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:51043)
0927 07:04:44.760879 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:44.760946 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 07:04:44.761916 30820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:44.761564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:56793)
0927 07:04:44.761727 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:44.761803 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 07:04:44.762099 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 30690
W20260927 07:04:44.762149 30818 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 07:04:44.876996 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:44.876136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:47899)
0927 07:04:44.876684 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:44.876773 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":285,"threads_started":1}
I20260927 07:04:45.326529 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:45.325655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:38187)
0927 07:04:45.326197 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:45.326288 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":280,"threads_started":1}
I20260927 07:04:45.573725 30824 raft_consensus.cc:482] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:04:45.573982 30824 raft_consensus.cc:504] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:45.575547 30824 leader_election.cc:288] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677), cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665)
Sep 27 07:04:45 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
Sep 27 07:04:45 dist-test-slave-p6hz krb5kdc[29237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:04:45.601157 30828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:45.576799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:42379 (local address 127.13.121.195:45665)
0927 07:04:45.578168 (+ 1369us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:45.578182 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:45.578574 (+ 392us) server_negotiation.cc:407] Connection header received
0927 07:04:45.580688 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:45.580698 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:45.581108 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:45.581437 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:45.583516 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:45.585388 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:45.588029 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:45.589039 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:45.589226 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:45.594018 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:45.594026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:45.594038 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:45.594074 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:45.597168 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:45.598125 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:45.598132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:45.598138 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:45.598312 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:45.598762 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:45.598766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:45.598769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:45.599399 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:45.599577 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:45.600377 (+ 800us) server_negotiation.cc:299] Negotiation successful
0927 07:04:45.600731 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1208,"thread_start_us":1090,"threads_started":1}
I20260927 07:04:45.601167 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:45.576055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:37329 (local address 127.13.121.194:42677)
0927 07:04:45.577920 (+ 1865us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:45.577935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:45.577984 (+ 49us) server_negotiation.cc:407] Connection header received
0927 07:04:45.580116 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:45.580129 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:45.580648 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:45.581048 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:45.583534 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:45.585388 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:45.588029 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:45.589039 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:45.589226 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:45.593822 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:45.593834 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:45.593857 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:45.593908 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:45.597153 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:45.598132 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:45.598138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:45.598143 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:45.598322 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:45.598767 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:45.598771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:45.598773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:45.599398 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:45.599576 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:45.600381 (+ 805us) server_negotiation.cc:299] Negotiation successful
0927 07:04:45.600778 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1678,"thread_start_us":1527,"threads_started":1}
I20260927 07:04:45.601280 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:45.575839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42677 (local address 127.13.121.193:37329)
0927 07:04:45.576417 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:45.576461 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:45.577089 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:45.581176 (+ 4087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:45.581185 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:45.582188 (+ 1003us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:04:45.583303 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:45.583331 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:45.585603 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:45.585610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:45.587875 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:45.587893 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:45.589234 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:45.589245 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:45.589597 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:45.590488 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:04:45.590507 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:04:45.593471 (+ 2964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:04:45.597344 (+ 3873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:45.597354 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:45.597368 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:45.597986 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:45.598436 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:45.598442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:45.598445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:45.598649 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:45.599538 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:04:45.599546 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:04:45.599858 (+ 312us) client_negotiation.cc:769] Sending connection context
0927 07:04:45.600144 (+ 286us) client_negotiation.cc:240] Negotiation successful
0927 07:04:45.600740 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":318,"spinlock_wait_cycles":8576,"thread_start_us":250,"threads_started":1}
I20260927 07:04:45.601421 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:45.576486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:45665 (local address 127.13.121.193:42379)
0927 07:04:45.578459 (+ 1973us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:45.578474 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 07:04:45.578612 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:04:45.581415 (+ 2803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:04:45.581419 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:04:45.582189 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:04:45.583303 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:45.583331 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:45.585607 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:45.585611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:45.587875 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:04:45.587893 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:04:45.589273 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:04:45.589280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:04:45.589593 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:45.590475 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:04:45.590515 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:04:45.593750 (+ 3235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:04:45.597344 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:45.597354 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:45.597368 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:45.597971 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:45.598424 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:04:45.598430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:04:45.598434 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:04:45.598639 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:04:45.599537 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:04:45.599546 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:04:45.599858 (+ 312us) client_negotiation.cc:769] Sending connection context
0927 07:04:45.600143 (+ 285us) client_negotiation.cc:240] Negotiation successful
0927 07:04:45.600895 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1878,"mutex_wait_us":539,"spinlock_wait_cycles":53760,"thread_start_us":1703,"threads_started":1}
W20260927 07:04:45.604836 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
W20260927 07:04:45.605144 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
I20260927 07:04:45.605257 30442 leader_election.cc:302] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c9c527218ce464da8a12134a73ea6cb; no voters: 947c8106699644f0b5058c2ea438aa2c, cfda194b51194355b9133bd3263bd255
I20260927 07:04:45.605659 30824 raft_consensus.cc:2726] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:04:45.763865 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:45.762758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:47987)
0927 07:04:45.763397 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:45.763585 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":245,"threads_started":1}
I20260927 07:04:45.878348 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:45.877913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:45901)
0927 07:04:45.878094 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:45.878198 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260927 07:04:46.328334 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:46.327520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:55931)
0927 07:04:46.328060 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:46.328142 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":295,"threads_started":1}
I20260927 07:04:46.765584 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:46.764893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:37293)
0927 07:04:46.765339 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:46.765427 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":219,"threads_started":1}
I20260927 07:04:46.880246 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:46.879355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:58505)
0927 07:04:46.879941 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:46.880029 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":297,"threads_started":1}
I20260927 07:04:47.307292 30833 raft_consensus.cc:482] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:04:47.307482 30833 raft_consensus.cc:504] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:47.308081 30833 leader_election.cc:288] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677), cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665)
W20260927 07:04:47.309106 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
W20260927 07:04:47.309341 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
I20260927 07:04:47.309494 30442 leader_election.cc:302] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c9c527218ce464da8a12134a73ea6cb; no voters: 947c8106699644f0b5058c2ea438aa2c, cfda194b51194355b9133bd3263bd255
I20260927 07:04:47.309736 30833 raft_consensus.cc:2726] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:04:47.329963 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:47.329268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:38943)
0927 07:04:47.329703 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:47.329792 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":234,"threads_started":1}
I20260927 07:04:47.767382 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:47.766527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:47385)
0927 07:04:47.767059 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:47.767207 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":292,"threads_started":1}
I20260927 07:04:47.882055 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:47.881306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:33759)
0927 07:04:47.881790 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:47.881882 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":241,"threads_started":1}
I20260927 07:04:48.331570 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:48.330802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:37209)
0927 07:04:48.331304 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:48.331398 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":217,"threads_started":1}
I20260927 07:04:48.768959 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:48.768280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:41217)
0927 07:04:48.768702 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:48.768778 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":211,"threads_started":1}
I20260927 07:04:48.884080 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:48.883322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:50181)
0927 07:04:48.883810 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:48.883897 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":236,"threads_started":1}
I20260927 07:04:49.288362 30840 raft_consensus.cc:482] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:04:49.288552 30840 raft_consensus.cc:504] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:49.289183 30840 leader_election.cc:288] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677), cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665)
W20260927 07:04:49.290171 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
W20260927 07:04:49.290405 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
I20260927 07:04:49.290515 30442 leader_election.cc:302] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c9c527218ce464da8a12134a73ea6cb; no voters: 947c8106699644f0b5058c2ea438aa2c, cfda194b51194355b9133bd3263bd255
I20260927 07:04:49.290745 30840 raft_consensus.cc:2726] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:04:49.333302 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:49.332548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:49089)
0927 07:04:49.333043 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:49.333129 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":254,"threads_started":1}
I20260927 07:04:49.770663 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:49.769955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:33849)
0927 07:04:49.770399 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:49.770491 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":227,"threads_started":1}
I20260927 07:04:49.885780 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:49.885006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:42369)
0927 07:04:49.885496 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:49.885596 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":268,"threads_started":1}
I20260927 07:04:50.335109 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:50.334313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:56651)
0927 07:04:50.334827 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:50.334920 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":266,"threads_started":1}
I20260927 07:04:50.772462 30845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:50.771669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:51613)
0927 07:04:50.772165 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:50.772255 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":252,"threads_started":1}
I20260927 07:04:50.887425 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:50.886681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:34867)
0927 07:04:50.887092 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:50.887217 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":205,"threads_started":1}
I20260927 07:04:51.337014 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:51.336181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:57973)
0927 07:04:51.336721 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:51.336824 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":241,"threads_started":1}
I20260927 07:04:51.774230 30848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:51.773417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:50161)
0927 07:04:51.773948 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:51.774036 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":251,"threads_started":1}
I20260927 07:04:51.889125 30849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:51.888358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:46381)
0927 07:04:51.888853 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:51.888948 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":255,"threads_started":1}
I20260927 07:04:52.338703 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:52.337999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:33251)
0927 07:04:52.338419 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:52.338516 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":212,"threads_started":1}
I20260927 07:04:52.776024 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:52.775242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:46511)
0927 07:04:52.775716 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:52.775828 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
I20260927 07:04:52.890794 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:52.890061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:48815)
0927 07:04:52.890522 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:52.890612 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":233,"threads_started":1}
I20260927 07:04:53.230365 30853 raft_consensus.cc:482] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:04:53.230602 30853 raft_consensus.cc:504] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:53.231353 30853 leader_election.cc:288] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677), cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665)
W20260927 07:04:53.232352 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
W20260927 07:04:53.232582 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
I20260927 07:04:53.232681 30442 leader_election.cc:302] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c9c527218ce464da8a12134a73ea6cb; no voters: 947c8106699644f0b5058c2ea438aa2c, cfda194b51194355b9133bd3263bd255
I20260927 07:04:53.232923 30853 raft_consensus.cc:2726] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:04:53.340423 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:53.339691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:49837)
0927 07:04:53.340174 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:53.340268 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":237,"threads_started":1}
I20260927 07:04:53.777688 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:53.776947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:60447)
0927 07:04:53.777448 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:53.777532 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":259,"threads_started":1}
I20260927 07:04:53.892686 30856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:53.891942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:37589)
0927 07:04:53.892425 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:53.892514 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":238,"threads_started":1}
I20260927 07:04:54.342216 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:54.341449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:53293)
0927 07:04:54.341933 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:54.342027 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":255,"threads_started":1}
I20260927 07:04:54.779424 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:54.778677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:53331)
0927 07:04:54.779108 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:54.779243 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":227,"threads_started":1}
I20260927 07:04:54.894395 30859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:54.893642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:38573)
0927 07:04:54.894089 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:54.894209 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":227,"threads_started":1}
I20260927 07:04:55.344094 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:55.343316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:42523)
0927 07:04:55.343827 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:55.343932 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":241,"threads_started":1}
I20260927 07:04:55.781100 30861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:55.780380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:56101)
0927 07:04:55.780847 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:55.780944 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":223,"threads_started":1}
I20260927 07:04:55.896013 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:55.895323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:53211)
0927 07:04:55.895755 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:55.895849 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":207,"threads_started":1}
I20260927 07:04:56.345871 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:56.345066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:42405)
0927 07:04:56.345556 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:56.345658 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":243,"threads_started":1}
I20260927 07:04:56.782899 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:56.782142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:58165)
0927 07:04:56.782609 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:56.782719 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
I20260927 07:04:56.897881 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:56.896941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:48307)
0927 07:04:56.897513 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:56.897636 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":293,"threads_started":1}
I20260927 07:04:57.347751 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:57.346848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:51789)
0927 07:04:57.347430 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:57.347569 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":325,"threads_started":1}
I20260927 07:04:57.508013 30867 raft_consensus.cc:482] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:04:57.508281 30867 raft_consensus.cc:504] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "947c8106699644f0b5058c2ea438aa2c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42677 } } peers { permanent_uuid: "4c9c527218ce464da8a12134a73ea6cb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 44001 } } peers { permanent_uuid: "cfda194b51194355b9133bd3263bd255" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 45665 } }
I20260927 07:04:57.508963 30867 leader_election.cc:288] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677), cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665)
W20260927 07:04:57.510044 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 947c8106699644f0b5058c2ea438aa2c (127.13.121.194:42677): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
W20260927 07:04:57.510300 30442 leader_election.cc:341] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cfda194b51194355b9133bd3263bd255 (127.13.121.195:45665): Not found: Tablet not found: 77aeda21cb8d43b693530db7a0dc4c2f
I20260927 07:04:57.510424 30442 leader_election.cc:302] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c9c527218ce464da8a12134a73ea6cb; no voters: 947c8106699644f0b5058c2ea438aa2c, cfda194b51194355b9133bd3263bd255
I20260927 07:04:57.510689 30867 raft_consensus.cc:2726] T 77aeda21cb8d43b693530db7a0dc4c2f P 4c9c527218ce464da8a12134a73ea6cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 07:04:57.784920 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:57.784005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:49691)
0927 07:04:57.784617 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:57.784741 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":342,"threads_started":1}
I20260927 07:04:57.899930 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:57.898952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:55207)
0927 07:04:57.899507 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:57.899623 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":280,"threads_started":1}
I20260927 07:04:58.349502 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:58.348691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:44773)
0927 07:04:58.349204 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:58.349311 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":249,"threads_started":1}
I20260927 07:04:58.786756 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:58.785915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:51497)
0927 07:04:58.786430 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:58.786533 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":268,"threads_started":1}
I20260927 07:04:58.901859 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:58.901103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.193:44845)
0927 07:04:58.901581 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:58.901687 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":246,"threads_started":1}
I20260927 07:04:59.351315 30873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:59.350551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.194:55101)
0927 07:04:59.350998 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:59.351101 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":220,"threads_started":1}
I20260927 07:04:59.765879 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 30385
I20260927 07:04:59.784330 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:59.767181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34884 (local address 127.13.121.193:44001)
0927 07:04:59.767702 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:59.767709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:59.767737 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:04:59.767812 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:59.767818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:59.768030 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:59.768171 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:59.769577 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:59.770593 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:59.771949 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:59.772588 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:59.772700 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:59.778783 (+ 6083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:59.778799 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:59.778817 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:59.778860 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:59.781656 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:59.782399 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:59.782406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:59.782408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:59.782557 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:59.782870 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:59.782874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:59.782876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:59.783322 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:59.783571 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:59.783825 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 07:04:59.784041 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":235,"threads_started":1}
I20260927 07:04:59.785561 30550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:04:59.788921 30876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:59.787964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:38269 (local address 127.13.121.195:43807)
0927 07:04:59.788533 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0927 07:04:59.788704 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.254:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":270,"threads_started":1}
I20260927 07:04:59.876278 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 30385
I20260927 07:04:59.890683 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 30558
I20260927 07:04:59.903481 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:04:59.891431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.13.121.194:42677)
0927 07:04:59.891965 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0927 07:04:59.891970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:04:59.891987 (+ 17us) server_negotiation.cc:407] Connection header received
0927 07:04:59.892044 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:04:59.892048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:04:59.892108 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:04:59.892201 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:04:59.893504 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:59.894298 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:59.895612 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:04:59.896058 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:04:59.896153 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:04:59.898483 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:04:59.898488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:04:59.898491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:04:59.898521 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:04:59.901071 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:59.901706 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:59.901710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:59.901713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:59.901868 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:04:59.902171 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:04:59.902174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:04:59.902176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:04:59.902447 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:04:59.902663 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 07:04:59.902941 (+ 278us) server_negotiation.cc:299] Negotiation successful
0927 07:04:59.903175 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":311,"threads_started":1}
I20260927 07:04:59.904172 30682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:04:59.986634 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 30558
I20260927 07:05:00.001077 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 30690
I20260927 07:05:00.014298 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:00.001787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.13.121.195:45665)
0927 07:05:00.002390 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:00.002395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:00.002417 (+ 22us) server_negotiation.cc:407] Connection header received
0927 07:05:00.002481 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:00.002484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:00.002562 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:00.002680 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:00.004116 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:00.004906 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:00.006181 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:00.006606 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:00.006716 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:00.009149 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:00.009162 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:00.009165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:00.009201 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:00.011790 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:00.012419 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:00.012423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:00.012426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:00.012574 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:00.012918 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:00.012922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:00.012924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:00.013193 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:00.013442 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:00.013721 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 07:05:00.013939 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":245,"threads_started":1}
I20260927 07:05:00.014986 30813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:00.098616 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 30690
2026-09-27T07:05:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64190 ms)
[----------] 30 tests from SecurityITest (293398 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:00 dist-test-slave-p6hz krb5kdc[30883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:00 dist-test-slave-p6hz krb5kdc[30883](info): set up 2 sockets
Sep 27 07:05:00 dist-test-slave-p6hz krb5kdc[30883](info): commencing operation
krb5kdc: starting...
W20260927 07:05:01.119375 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:01 dist-test-slave-p6hz krb5kdc[30883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:01.358901 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:36009
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:36009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:01.691013 30896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:01.691377 30896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:01.691522 30896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:01.699082 30896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:01.699251 30896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:01.699334 30896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:01.699409 30896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:01.699470 30896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:01.712873 30896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:36009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:36009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:01.714653 30896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:01.716420 30896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:01.731778 30901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:01.734015 30902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:01.734556 30896 server_base.cc:1034] running on GCE node
W20260927 07:05:01.734685 30904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:01.735726 30896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:01.737694 30896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:01.738941 30896 hybrid_clock.cc:648] HybridClock initialized: now 1790492701738878 us; error 55 us; skew 500 ppm
I20260927 07:05:01.739395 30896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:01 dist-test-slave-p6hz krb5kdc[30883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492701, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:01.742753 30896 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:01.744482 30896 webserver.cc:460] Webserver started at http://127.13.121.254:34975/ using document root <none> and password file <none>
I20260927 07:05:01.745195 30896 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:01.745322 30896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:01.745617 30896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:01.748850 30896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "509d9f2bdc174e9bb2ccf9119f0eb411"
format_stamp: "Formatted at 2026-09-27 07:05:01 on dist-test-slave-p6hz"
server_key: "350940206eb32f43285b76d88e0ab148"
server_key_iv: "d79058f79c48a4fdef1fe83dac0b79ea"
server_key_version: "encryptionkey@0"
I20260927 07:05:01.749620 30896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "509d9f2bdc174e9bb2ccf9119f0eb411"
format_stamp: "Formatted at 2026-09-27 07:05:01 on dist-test-slave-p6hz"
server_key: "350940206eb32f43285b76d88e0ab148"
server_key_iv: "d79058f79c48a4fdef1fe83dac0b79ea"
server_key_version: "encryptionkey@0"
I20260927 07:05:01.755187 30896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:05:01.758975 30911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:01.760454 30896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260927 07:05:01.760643 30896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "509d9f2bdc174e9bb2ccf9119f0eb411"
format_stamp: "Formatted at 2026-09-27 07:05:01 on dist-test-slave-p6hz"
server_key: "350940206eb32f43285b76d88e0ab148"
server_key_iv: "d79058f79c48a4fdef1fe83dac0b79ea"
server_key_version: "encryptionkey@0"
I20260927 07:05:01.760842 30896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:01.778887 30896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:01.783216 30896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:01.783495 30896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:01.801518 30896 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:36009
I20260927 07:05:01.801525 30962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:36009 every 8 connection(s)
I20260927 07:05:01.802897 30896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:01.806882 30963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:01.812341 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 30896
I20260927 07:05:01.812554 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:01.813333 13799 external_mini_cluster.cc:1428] Setting key 1f236a0a4499056902715cf2a4209b62
I20260927 07:05:01.819095 30963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411: Bootstrap starting.
I20260927 07:05:01.823308 30963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:01.824764 30963 log.cc:824] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:01.828660 30963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411: No bootstrap required, opened a new log
Sep 27 07:05:01 dist-test-slave-p6hz krb5kdc[30883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:01.837793 30963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d9f2bdc174e9bb2ccf9119f0eb411" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36009 } }
I20260927 07:05:01.838335 30963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:01.838459 30963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 509d9f2bdc174e9bb2ccf9119f0eb411, State: Initialized, Role: FOLLOWER
I20260927 07:05:01.839167 30963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d9f2bdc174e9bb2ccf9119f0eb411" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36009 } }
I20260927 07:05:01.839430 30963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:01.839563 30963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:01.839725 30963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:01.840116 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:01.815881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47872 (local address 127.13.121.254:36009)
0927 07:05:01.816814 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:01.816827 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:01.816877 (+ 50us) server_negotiation.cc:407] Connection header received
0927 07:05:01.818702 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:01.818731 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:01.819267 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:01.819676 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:01.821723 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:01.823245 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:01.825276 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:01.826483 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:01.826643 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:01.831834 (+ 5191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:01.831846 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:01.831864 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:01.831915 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:01.835079 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:01.835969 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:01.835979 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:01.835986 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:01.836172 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:01.836755 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:01.836763 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:01.836765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:01.837575 (+ 810us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:01.837878 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:01.838419 (+ 541us) server_negotiation.cc:299] Negotiation successful
0927 07:05:01.838923 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":414,"threads_started":1}
I20260927 07:05:01.842147 30963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d9f2bdc174e9bb2ccf9119f0eb411" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36009 } }
I20260927 07:05:01.842665 30963 leader_election.cc:302] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 509d9f2bdc174e9bb2ccf9119f0eb411; no voters:
I20260927 07:05:01.843307 30963 leader_election.cc:288] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:01.843415 30968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:01.844045 30968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [term 1 LEADER]: Becoming Leader. State: Replica: 509d9f2bdc174e9bb2ccf9119f0eb411, State: Running, Role: LEADER
I20260927 07:05:01.844564 30968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d9f2bdc174e9bb2ccf9119f0eb411" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36009 } }
I20260927 07:05:01.845290 30963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:01.847105 30970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 509d9f2bdc174e9bb2ccf9119f0eb411. Latest consensus state: current_term: 1 leader_uuid: "509d9f2bdc174e9bb2ccf9119f0eb411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d9f2bdc174e9bb2ccf9119f0eb411" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36009 } } }
I20260927 07:05:01.847078 30969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "509d9f2bdc174e9bb2ccf9119f0eb411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d9f2bdc174e9bb2ccf9119f0eb411" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 36009 } } }
I20260927 07:05:01.847585 30970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:01.847585 30969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:01.849653 30974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:01.854434 30974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:01.863523 30974 catalog_manager.cc:1347] Generated new cluster ID: d16f398b3e8048f581bb4746ba24f176
I20260927 07:05:01.863684 30974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:01.888979 30974 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:01.890372 30974 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:01.913578 30974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411: Generated new TSK 0
I20260927 07:05:01.914834 30974 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:05:01.995986 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36009
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:02.385135 30989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:02.385439 30989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:02.385536 30989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:02.392848 30989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:02.392970 30989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:02.393097 30989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:05:02.402931 30989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:02.404335 30989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:02.405717 30989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:02.419581 30994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:02.419389 30997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:02.419432 30995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:02.420198 30989 server_base.cc:1034] running on GCE node
I20260927 07:05:02.420842 30989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:02.421857 30989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:02.423075 30989 hybrid_clock.cc:648] HybridClock initialized: now 1790492702422985 us; error 70 us; skew 500 ppm
I20260927 07:05:02.423434 30989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:02 dist-test-slave-p6hz krb5kdc[30883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:02.426534 30989 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:05:02.428079 30989 webserver.cc:460] Webserver started at http://127.13.121.193:36897/ using document root <none> and password file <none>
I20260927 07:05:02.428804 30989 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:02.428930 30989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:02.429247 30989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:02.432348 30989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "388dba9cd7994569b561dc6683e93afe"
format_stamp: "Formatted at 2026-09-27 07:05:02 on dist-test-slave-p6hz"
server_key: "e4674a4d244ab4e97025a029f3b03901"
server_key_iv: "5ef2721060bf9664c3024329ca54b58a"
server_key_version: "encryptionkey@0"
I20260927 07:05:02.433139 30989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "388dba9cd7994569b561dc6683e93afe"
format_stamp: "Formatted at 2026-09-27 07:05:02 on dist-test-slave-p6hz"
server_key: "e4674a4d244ab4e97025a029f3b03901"
server_key_iv: "5ef2721060bf9664c3024329ca54b58a"
server_key_version: "encryptionkey@0"
I20260927 07:05:02.438563 30989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:05:02.442377 31004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:02.443854 30989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:02.444021 30989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "388dba9cd7994569b561dc6683e93afe"
format_stamp: "Formatted at 2026-09-27 07:05:02 on dist-test-slave-p6hz"
server_key: "e4674a4d244ab4e97025a029f3b03901"
server_key_iv: "5ef2721060bf9664c3024329ca54b58a"
server_key_version: "encryptionkey@0"
I20260927 07:05:02.444231 30989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:02.481904 30989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:02.486505 30989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:02.486821 30989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:02.487919 30989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:02.489331 30989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:02.489408 30989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:02.489532 30989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:02.489588 30989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:02.519563 30989 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:35793
I20260927 07:05:02.519624 31116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:35793 every 8 connection(s)
I20260927 07:05:02.521036 30989 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:05:02.525249 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 30989
I20260927 07:05:02.525504 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:05:02.526358 13799 external_mini_cluster.cc:1428] Setting key ce4d60670e609ec35a0f8a03d99a132b
Sep 27 07:05:02 dist-test-slave-p6hz krb5kdc[30883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:02.545696 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:02.524053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:42349 (local address 127.13.121.254:36009)
0927 07:05:02.525769 (+ 1716us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:02.525776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:02.526819 (+ 1043us) server_negotiation.cc:407] Connection header received
0927 07:05:02.529401 (+ 2582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:02.529407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:02.529496 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:02.529595 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:02.531344 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:02.532091 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:02.533935 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:02.534375 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:02.534497 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:02.538996 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:02.539003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:02.539006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:02.539048 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:02.541744 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:02.542859 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:02.542864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:02.542866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:02.543021 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:02.543639 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:02.543644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:02.543646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:02.543954 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:02.544081 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:02.545118 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0927 07:05:02.545323 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1560,"thread_start_us":3002,"threads_started":1}
I20260927 07:05:02.547207 31120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:02.524748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36009 (local address 127.13.121.193:42349)
0927 07:05:02.526639 (+ 1891us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:02.526686 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:02.529055 (+ 2369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:02.529884 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:02.529893 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:02.530494 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:02.531111 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:02.531130 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:02.532278 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:02.532283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:02.533758 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:02.533774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:02.534636 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:02.534643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:02.534904 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:02.535648 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:02.535678 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:02.538654 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:02.541959 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:02.541970 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:02.541992 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:02.542689 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:02.543178 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:02.543183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:02.543186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:02.543478 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:02.544081 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:02.544091 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:02.544626 (+ 535us) client_negotiation.cc:769] Sending connection context
0927 07:05:02.544952 (+ 326us) client_negotiation.cc:240] Negotiation successful
0927 07:05:02.545640 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1616,"thread_start_us":1405,"threads_started":1}
I20260927 07:05:02.549013 31117 heartbeater.cc:347] Connected to a master server at 127.13.121.254:36009
I20260927 07:05:02.549329 31117 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:02.550148 31117 heartbeater.cc:511] Master 127.13.121.254:36009 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:05:02.552441 30928 ts_manager.cc:194] Registered new tserver with Master: 388dba9cd7994569b561dc6683e93afe (127.13.121.193:35793)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
I20260927 07:05:02.554255 30928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:42349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:05:02.583065 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36009
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:02.922034 31123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:02.922356 31123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:02.922497 31123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:02.930595 31123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:02.930752 31123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:02.930908 31123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:05:02.941182 31123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:02.942525 31123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:02.943892 31123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:02.957391 31128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:02.957361 31131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:02.957417 31129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:02.958622 31123 server_base.cc:1034] running on GCE node
I20260927 07:05:02.959375 31123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:02.960453 31123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:02.961683 31123 hybrid_clock.cc:648] HybridClock initialized: now 1790492702961624 us; error 44 us; skew 500 ppm
I20260927 07:05:02.962010 31123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:02 dist-test-slave-p6hz krb5kdc[30883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:02.965312 31123 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:05:02.966922 31123 webserver.cc:460] Webserver started at http://127.13.121.194:32813/ using document root <none> and password file <none>
I20260927 07:05:02.967698 31123 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:02.967823 31123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:02.968134 31123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:02.971187 31123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "94cf0340b8af4e6185007d53d4a4a6eb"
format_stamp: "Formatted at 2026-09-27 07:05:02 on dist-test-slave-p6hz"
server_key: "6a7a906d2f14e37d96ca7b774d2dc58e"
server_key_iv: "7ce94f666cbee175135f9c2ba65fe099"
server_key_version: "encryptionkey@0"
I20260927 07:05:02.972024 31123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "94cf0340b8af4e6185007d53d4a4a6eb"
format_stamp: "Formatted at 2026-09-27 07:05:02 on dist-test-slave-p6hz"
server_key: "6a7a906d2f14e37d96ca7b774d2dc58e"
server_key_iv: "7ce94f666cbee175135f9c2ba65fe099"
server_key_version: "encryptionkey@0"
I20260927 07:05:02.977599 31123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:05:02.981437 31138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:02.982920 31123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:05:02.983121 31123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "94cf0340b8af4e6185007d53d4a4a6eb"
format_stamp: "Formatted at 2026-09-27 07:05:02 on dist-test-slave-p6hz"
server_key: "6a7a906d2f14e37d96ca7b774d2dc58e"
server_key_iv: "7ce94f666cbee175135f9c2ba65fe099"
server_key_version: "encryptionkey@0"
I20260927 07:05:02.983392 31123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:03.013984 31123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:03.018873 31123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:03.019202 31123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:03.020256 31123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:03.021684 31123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:03.021766 31123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:03.021894 31123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:03.021955 31123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:03.054970 31123 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:35917
I20260927 07:05:03.055011 31250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:35917 every 8 connection(s)
I20260927 07:05:03.056411 31123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:05:03.057571 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 31123
I20260927 07:05:03.057760 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:05:03.058239 13799 external_mini_cluster.cc:1428] Setting key 4050ba47053ec957bce0515d6707efa4
Sep 27 07:05:03 dist-test-slave-p6hz krb5kdc[30883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:03.079387 31253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.059891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:49577 (local address 127.13.121.254:36009)
0927 07:05:03.060338 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.060343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.061706 (+ 1363us) server_negotiation.cc:407] Connection header received
0927 07:05:03.063686 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.063692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.063780 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.063877 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:03.065712 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.066452 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.068263 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.068759 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.068899 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.073150 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:03.073157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:03.073160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:03.073200 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:03.075888 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:03.076731 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:03.076735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:03.076738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:03.076873 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:03.077324 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:03.077328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:03.077330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:03.077638 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:03.077800 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.078832 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.079033 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":233,"threads_started":1}
I20260927 07:05:03.080904 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.060678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36009 (local address 127.13.121.194:49577)
0927 07:05:03.061534 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:03.061586 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:03.063362 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:03.064200 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:03.064210 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:03.064856 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:03.065513 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.065535 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.066630 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.066635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.068017 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.068036 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.069034 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.069041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.069362 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.070128 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:03.070158 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:03.072861 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:03.076076 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:03.076086 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:03.076102 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:03.076584 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:03.076986 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:03.076991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:03.076995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:03.077204 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:03.077823 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:03.077834 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:03.078361 (+ 527us) client_negotiation.cc:769] Sending connection context
0927 07:05:03.078702 (+ 341us) client_negotiation.cc:240] Negotiation successful
0927 07:05:03.079368 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":329,"threads_started":1}
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:05:03.082762 31251 heartbeater.cc:347] Connected to a master server at 127.13.121.254:36009
I20260927 07:05:03.083073 31251 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:03.083889 31251 heartbeater.cc:511] Master 127.13.121.254:36009 requested a full tablet report, sending...
I20260927 07:05:03.085602 30928 ts_manager.cc:194] Registered new tserver with Master: 94cf0340b8af4e6185007d53d4a4a6eb (127.13.121.194:35917)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
I20260927 07:05:03.086522 30928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:49577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:05:03.115617 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36009
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:03.454071 31257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:03.454386 31257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:03.454532 31257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:03.464177 31257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:03.464321 31257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:03.464485 31257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:05:03.475246 31257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:36009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:03.476765 31257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:03.478310 31257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:03.492697 31263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:03.492647 31265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:03.492874 31262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:03.493546 31257 server_base.cc:1034] running on GCE node
I20260927 07:05:03.494175 31257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:03.495235 31257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:03.496467 31257 hybrid_clock.cc:648] HybridClock initialized: now 1790492703496407 us; error 50 us; skew 500 ppm
I20260927 07:05:03.496803 31257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:03 dist-test-slave-p6hz krb5kdc[30883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492703, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:03.499908 31257 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:05:03.501451 31257 webserver.cc:460] Webserver started at http://127.13.121.195:34373/ using document root <none> and password file <none>
I20260927 07:05:03.502127 31257 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:03.502246 31257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:03.502538 31257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:03.505533 31257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "e0f6f8e2af9749f780da9a93b6f6f052"
format_stamp: "Formatted at 2026-09-27 07:05:03 on dist-test-slave-p6hz"
server_key: "c46d462ee563c6265fa2cd3fa1b5465b"
server_key_iv: "d6aef54f7c5a6f5ad641f43506c89319"
server_key_version: "encryptionkey@0"
I20260927 07:05:03.506330 31257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "e0f6f8e2af9749f780da9a93b6f6f052"
format_stamp: "Formatted at 2026-09-27 07:05:03 on dist-test-slave-p6hz"
server_key: "c46d462ee563c6265fa2cd3fa1b5465b"
server_key_iv: "d6aef54f7c5a6f5ad641f43506c89319"
server_key_version: "encryptionkey@0"
I20260927 07:05:03.512037 31257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:05:03.516089 31272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:03.518064 31257 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 07:05:03.518290 31257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "e0f6f8e2af9749f780da9a93b6f6f052"
format_stamp: "Formatted at 2026-09-27 07:05:03 on dist-test-slave-p6hz"
server_key: "c46d462ee563c6265fa2cd3fa1b5465b"
server_key_iv: "d6aef54f7c5a6f5ad641f43506c89319"
server_key_version: "encryptionkey@0"
I20260927 07:05:03.518534 31257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:03.537185 31257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:03.541918 31257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:03.542225 31257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:03.543323 31257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:03.544730 31257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:03.544809 31257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:03.544941 31257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:03.545002 31257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:03.558046 31117 heartbeater.cc:503] Master 127.13.121.254:36009 was elected leader, sending a full tablet report...
I20260927 07:05:03.576521 31257 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:34473
I20260927 07:05:03.576570 31384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:34473 every 8 connection(s)
I20260927 07:05:03.577768 31257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:05:03.580293 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 31257
I20260927 07:05:03.580478 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:05:03.581007 13799 external_mini_cluster.cc:1428] Setting key ee476c04cf49ec0c7588e7158b9f6c71
Sep 27 07:05:03 dist-test-slave-p6hz krb5kdc[30883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492703, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:03.598946 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.581519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:49011 (local address 127.13.121.254:36009)
0927 07:05:03.582354 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.582360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.582379 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:05:03.584450 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.584456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.584542 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.584631 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:03.586206 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.587072 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.588804 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.589253 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.589353 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.592915 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:03.592922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:03.592925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:03.592962 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:03.595699 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:03.596508 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:03.596512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:03.596514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:03.596638 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:03.597117 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:03.597122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:03.597125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:03.597442 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:03.597573 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.598306 (+ 733us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.598546 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":858,"threads_started":1}
I20260927 07:05:03.599942 31387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.580982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:36009 (local address 127.13.121.195:49011)
0927 07:05:03.581863 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:03.581903 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:03.584124 (+ 2221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:03.584836 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:03.584845 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:03.585426 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:03.586009 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.586027 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.587283 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.587288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.588650 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.588664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.589452 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.589457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.589662 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.590317 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:03.590347 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:03.592676 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:03.595830 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:03.595838 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:03.595854 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:03.596403 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:03.596773 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:03.596778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:03.596782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:03.597002 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:03.597568 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:03.597576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:03.597950 (+ 374us) client_negotiation.cc:769] Sending connection context
0927 07:05:03.598203 (+ 253us) client_negotiation.cc:240] Negotiation successful
0927 07:05:03.598751 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":548,"threads_started":1}
I20260927 07:05:03.601358 31385 heartbeater.cc:347] Connected to a master server at 127.13.121.254:36009
I20260927 07:05:03.601661 31385 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:03.602386 31385 heartbeater.cc:511] Master 127.13.121.254:36009 requested a full tablet report, sending...
I20260927 07:05:03.603817 30928 ts_manager.cc:194] Registered new tserver with Master: e0f6f8e2af9749f780da9a93b6f6f052 (127.13.121.195:34473)
I20260927 07:05:03.604624 30928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:49011
I20260927 07:05:03.609239 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:05:03.627096 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.614009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47878 (local address 127.13.121.254:36009)
0927 07:05:03.614195 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.614199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.614645 (+ 446us) server_negotiation.cc:407] Connection header received
0927 07:05:03.614764 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.614769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.614853 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.614944 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:03.616433 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.617196 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.618691 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.619118 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.619269 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.621678 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:03.621683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:03.621687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:03.621727 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:03.624387 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:03.625167 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:03.625171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:03.625174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:03.625315 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:03.625653 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:03.625657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:03.625660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:03.626032 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:03.626220 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.626492 (+ 272us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.626685 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 07:05:03.631268 30928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:05:03.638183 30928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:05:03.662181 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.651573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39794 (local address 127.13.121.194:35917)
0927 07:05:03.653535 (+ 1962us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.653556 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.653590 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:05:03.653704 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.653713 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.653981 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.654179 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:03.654895 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.656492 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.659349 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.660920 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.661089 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.661274 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.661655 (+ 381us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.661837 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1791,"thread_start_us":1688,"threads_started":1}
I20260927 07:05:03.662209 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.651104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:35917 (local address 127.0.0.1:39794)
0927 07:05:03.652155 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:03.652200 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:03.652363 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:03.654212 (+ 1849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:03.654220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:03.654253 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:03.654682 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.654702 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.656637 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.656642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.659176 (+ 2534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.659193 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.661133 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.661140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.661357 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.661421 (+ 64us) client_negotiation.cc:769] Sending connection context
0927 07:05:03.661580 (+ 159us) client_negotiation.cc:240] Negotiation successful
0927 07:05:03.661786 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":640,"threads_started":1}
I20260927 07:05:03.662642 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.651564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:34473 (local address 127.0.0.1:45038)
0927 07:05:03.652921 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:03.652938 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:03.653052 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:03.654548 (+ 1496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:03.654553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:03.654578 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:03.655584 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.655599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.657324 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.657330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.659497 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.659510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.661486 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.661492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.661675 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.661705 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 07:05:03.661779 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 07:05:03.661888 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1246,"mutex_wait_us":9,"thread_start_us":1029,"threads_started":1}
I20260927 07:05:03.663084 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.652609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:35793 (local address 127.0.0.1:58794)
0927 07:05:03.655323 (+ 2714us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:03.655339 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:03.655445 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:03.656424 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:03.656427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:03.656447 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:03.656879 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.656890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.658289 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.658293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.660204 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.660220 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.662220 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.662227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.662404 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.662435 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 07:05:03.662530 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 07:05:03.662667 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2593,"mutex_wait_us":3,"thread_start_us":2424,"threads_started":1}
I20260927 07:05:03.663115 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.653199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.13.121.193:35793)
0927 07:05:03.654845 (+ 1646us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.654854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.655598 (+ 744us) server_negotiation.cc:407] Connection header received
0927 07:05:03.655704 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.655713 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.655989 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.656181 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:03.657060 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.658139 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.660398 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.661693 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.661877 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.662032 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.662610 (+ 578us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.662776 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1484,"thread_start_us":1311,"threads_started":1}
I20260927 07:05:03.664017 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.652237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45038 (local address 127.13.121.195:34473)
0927 07:05:03.653971 (+ 1734us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.653977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.654005 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:05:03.654086 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.654091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.654319 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.655243 (+ 924us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:03.655812 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.657180 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.659701 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.660989 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.661174 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.661344 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.663526 (+ 2182us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.663706 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1581,"thread_start_us":1433,"threads_started":1}
I20260927 07:05:03.666952 31052 tablet_service.cc:1467] Processing CreateTablet for tablet ab53ef789cf94afaaf9265e975b3554a (DEFAULT_TABLE table=test-table [id=36c1dabd58cd4e3ca8e21c1f6c30049d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:03.666787 31186 tablet_service.cc:1467] Processing CreateTablet for tablet ab53ef789cf94afaaf9265e975b3554a (DEFAULT_TABLE table=test-table [id=36c1dabd58cd4e3ca8e21c1f6c30049d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:03.667703 31320 tablet_service.cc:1467] Processing CreateTablet for tablet ab53ef789cf94afaaf9265e975b3554a (DEFAULT_TABLE table=test-table [id=36c1dabd58cd4e3ca8e21c1f6c30049d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:03.668219 31052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab53ef789cf94afaaf9265e975b3554a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:03.668280 31186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab53ef789cf94afaaf9265e975b3554a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:03.669003 31320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab53ef789cf94afaaf9265e975b3554a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:03.679881 31404 tablet_bootstrap.cc:492] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: Bootstrap starting.
I20260927 07:05:03.682911 31405 tablet_bootstrap.cc:492] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: Bootstrap starting.
I20260927 07:05:03.684921 31404 tablet_bootstrap.cc:654] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:03.686307 31404 log.cc:824] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:03.686645 31405 tablet_bootstrap.cc:654] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:03.688176 31406 tablet_bootstrap.cc:492] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: Bootstrap starting.
I20260927 07:05:03.688901 31405 log.cc:824] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:03.691884 31405 tablet_bootstrap.cc:492] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: No bootstrap required, opened a new log
I20260927 07:05:03.691892 31404 tablet_bootstrap.cc:492] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: No bootstrap required, opened a new log
I20260927 07:05:03.692188 31404 ts_tablet_manager.cc:1397] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260927 07:05:03.692188 31405 ts_tablet_manager.cc:1397] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260927 07:05:03.692209 31406 tablet_bootstrap.cc:654] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:03.693218 31406 log.cc:824] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:03.695734 31406 tablet_bootstrap.cc:492] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: No bootstrap required, opened a new log
I20260927 07:05:03.695977 31406 ts_tablet_manager.cc:1397] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260927 07:05:03.702840 31405 raft_consensus.cc:348] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } }
I20260927 07:05:03.702840 31404 raft_consensus.cc:348] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } }
I20260927 07:05:03.703397 31404 raft_consensus.cc:374] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:03.703400 31405 raft_consensus.cc:374] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:03.703506 31404 raft_consensus.cc:729] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388dba9cd7994569b561dc6683e93afe, State: Initialized, Role: FOLLOWER
I20260927 07:05:03.703503 31405 raft_consensus.cc:729] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94cf0340b8af4e6185007d53d4a4a6eb, State: Initialized, Role: FOLLOWER
I20260927 07:05:03.703253 31406 raft_consensus.cc:348] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } }
I20260927 07:05:03.703665 31406 raft_consensus.cc:374] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:03.703768 31406 raft_consensus.cc:729] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0f6f8e2af9749f780da9a93b6f6f052, State: Initialized, Role: FOLLOWER
I20260927 07:05:03.704305 31406 consensus_queue.cc:260] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } }
I20260927 07:05:03.704298 31405 consensus_queue.cc:260] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } }
I20260927 07:05:03.704298 31404 consensus_queue.cc:260] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } }
I20260927 07:05:03.705394 31385 heartbeater.cc:503] Master 127.13.121.254:36009 was elected leader, sending a full tablet report...
I20260927 07:05:03.706143 31406 ts_tablet_manager.cc:1428] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260927 07:05:03.707782 31404 ts_tablet_manager.cc:1428] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260927 07:05:03.708046 31405 ts_tablet_manager.cc:1428] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260927 07:05:03.708781 31251 heartbeater.cc:503] Master 127.13.121.254:36009 was elected leader, sending a full tablet report...
I20260927 07:05:03.712402 31411 raft_consensus.cc:482] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:05:03.712702 31411 raft_consensus.cc:504] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } }
I20260927 07:05:03.714306 31411 leader_election.cc:288] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0f6f8e2af9749f780da9a93b6f6f052 (127.13.121.195:34473), 388dba9cd7994569b561dc6683e93afe (127.13.121.193:35793)
I20260927 07:05:03.720764 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.714618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:45077 (local address 127.13.121.195:34473)
0927 07:05:03.714821 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.714827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.715611 (+ 784us) server_negotiation.cc:407] Connection header received
0927 07:05:03.715779 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.715784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.715843 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.715912 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:03.716476 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.717214 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.718989 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.719818 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.719965 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.720040 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.720383 (+ 343us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.720504 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:05:03.720772 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.714738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:49149 (local address 127.13.121.193:35793)
0927 07:05:03.714937 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.714942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.715686 (+ 744us) server_negotiation.cc:407] Connection header received
0927 07:05:03.715860 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.715865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.715928 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.715999 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:03.716570 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.717322 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.719038 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.719876 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.720022 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.720105 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.720413 (+ 308us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.720540 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:05:03.721052 31413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.714508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:34473 (local address 127.13.121.194:45077)
0927 07:05:03.715507 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:03.715523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:03.715640 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:03.715940 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:03.715945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:03.715968 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:03.716343 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.716352 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.717337 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.717343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.718842 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.718854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.720039 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.720045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.720233 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.720265 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:05:03.720345 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 07:05:03.720483 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":19,"thread_start_us":676,"threads_started":1}
I20260927 07:05:03.721148 31414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.714748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:35793 (local address 127.13.121.194:49149)
0927 07:05:03.715601 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:03.715619 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:03.715742 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:03.716024 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:03.716028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:03.716065 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:03.716424 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.716433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.717459 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.717465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.718883 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:03.718895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.720085 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.720091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:03.720277 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.720300 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 07:05:03.720376 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 07:05:03.720498 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":24,"thread_start_us":478,"threads_started":1}
I20260927 07:05:03.722064 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab53ef789cf94afaaf9265e975b3554a" candidate_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" is_pre_election: true
I20260927 07:05:03.722061 31072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab53ef789cf94afaaf9265e975b3554a" candidate_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388dba9cd7994569b561dc6683e93afe" is_pre_election: true
I20260927 07:05:03.722647 31340 raft_consensus.cc:2454] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94cf0340b8af4e6185007d53d4a4a6eb in term 0.
I20260927 07:05:03.722658 31072 raft_consensus.cc:2454] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94cf0340b8af4e6185007d53d4a4a6eb in term 0.
I20260927 07:05:03.723376 31142 leader_election.cc:302] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388dba9cd7994569b561dc6683e93afe, 94cf0340b8af4e6185007d53d4a4a6eb; no voters:
I20260927 07:05:03.723800 31411 raft_consensus.cc:2781] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:05:03.723956 31411 raft_consensus.cc:482] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:05:03.724054 31411 raft_consensus.cc:3037] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:03.726218 31411 raft_consensus.cc:504] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } }
I20260927 07:05:03.726845 31411 leader_election.cc:288] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [CANDIDATE]: Term 1 election: Requested vote from peers e0f6f8e2af9749f780da9a93b6f6f052 (127.13.121.195:34473), 388dba9cd7994569b561dc6683e93afe (127.13.121.193:35793)
I20260927 07:05:03.727353 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab53ef789cf94afaaf9265e975b3554a" candidate_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f6f8e2af9749f780da9a93b6f6f052"
I20260927 07:05:03.727423 31072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab53ef789cf94afaaf9265e975b3554a" candidate_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388dba9cd7994569b561dc6683e93afe"
I20260927 07:05:03.727542 31340 raft_consensus.cc:3037] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:03.727643 31072 raft_consensus.cc:3037] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:03.729797 31340 raft_consensus.cc:2454] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cf0340b8af4e6185007d53d4a4a6eb in term 1.
I20260927 07:05:03.730058 31072 raft_consensus.cc:2454] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94cf0340b8af4e6185007d53d4a4a6eb in term 1.
I20260927 07:05:03.730216 31141 leader_election.cc:302] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94cf0340b8af4e6185007d53d4a4a6eb, e0f6f8e2af9749f780da9a93b6f6f052; no voters:
I20260927 07:05:03.730525 31411 raft_consensus.cc:2781] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:03.730949 31411 raft_consensus.cc:686] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 1 LEADER]: Becoming Leader. State: Replica: 94cf0340b8af4e6185007d53d4a4a6eb, State: Running, Role: LEADER
I20260927 07:05:03.731459 31411 consensus_queue.cc:237] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } }
I20260927 07:05:03.736001 30926 catalog_manager.cc:5495] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 94cf0340b8af4e6185007d53d4a4a6eb (127.13.121.194). New cstate: current_term: 1 leader_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94cf0340b8af4e6185007d53d4a4a6eb" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 35917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 } health_report { overall_health: UNKNOWN } } }
I20260927 07:05:03.771075 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.765567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.13.121.194:35917)
0927 07:05:03.765787 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.765794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.765869 (+ 75us) server_negotiation.cc:407] Connection header received
0927 07:05:03.766113 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.766117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.766199 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.766333 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:03.766710 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.767518 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.768910 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.769309 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.769414 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.769941 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:03.770464 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:03.770593 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.770705 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.770790 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260927 07:05:03.773595 31118 tablet.cc:2369] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:05:03.779212 31072 raft_consensus.cc:1261] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [term 1 FOLLOWER]: Refusing update from remote peer 94cf0340b8af4e6185007d53d4a4a6eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:03.779239 31340 raft_consensus.cc:1261] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [term 1 FOLLOWER]: Refusing update from remote peer 94cf0340b8af4e6185007d53d4a4a6eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:03.780262 31411 consensus_queue.cc:1035] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0f6f8e2af9749f780da9a93b6f6f052" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 34473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:03.780560 31415 consensus_queue.cc:1035] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "388dba9cd7994569b561dc6683e93afe" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 35793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:03.793113 31418 mvcc.cc:205] Tried to move back new op lower bound from 7333858114669993984 to 7333858114488025088. Current Snapshot: MvccSnapshot[applied={T|T < 7333858114669993984}]
I20260927 07:05:03.794317 31419 mvcc.cc:205] Tried to move back new op lower bound from 7333858114669993984 to 7333858114488025088. Current Snapshot: MvccSnapshot[applied={T|T < 7333858114669993984}]
I20260927 07:05:03.799494 31420 mvcc.cc:205] Tried to move back new op lower bound from 7333858114669993984 to 7333858114488025088. Current Snapshot: MvccSnapshot[applied={T|T < 7333858114669993984}]
I20260927 07:05:03.807184 30927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:03.807523 30927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:03.812407 30927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 509d9f2bdc174e9bb2ccf9119f0eb411: Sending DeleteTablet for 3 replicas of tablet ab53ef789cf94afaaf9265e975b3554a
I20260927 07:05:03.813585 31186 tablet_service.cc:1514] Processing DeleteTablet for tablet ab53ef789cf94afaaf9265e975b3554a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:03 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:39794
I20260927 07:05:03.813578 31052 tablet_service.cc:1514] Processing DeleteTablet for tablet ab53ef789cf94afaaf9265e975b3554a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:03 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:58794
I20260927 07:05:03.814031 31320 tablet_service.cc:1514] Processing DeleteTablet for tablet ab53ef789cf94afaaf9265e975b3554a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:03 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:45038
I20260927 07:05:03.816493 31051 tablet_service.cc:1514] Processing DeleteTablet for tablet ab53ef789cf94afaaf9265e975b3554a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:03 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:58794
I20260927 07:05:03.817023 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 30989
I20260927 07:05:03.829643 31430 tablet_replica.cc:330] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: stopping tablet replica
I20260927 07:05:03.830327 31429 tablet_replica.cc:330] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: stopping tablet replica
I20260927 07:05:03.830621 31430 raft_consensus.cc:2229] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:03.830739 31429 raft_consensus.cc:2229] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:03.831305 31430 raft_consensus.cc:2258] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:03.831333 31429 raft_consensus.cc:2258] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:03.832312 31428 tablet_replica.cc:330] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: stopping tablet replica
I20260927 07:05:03.832755 31428 raft_consensus.cc:2229] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:05:03.833535 31428 raft_consensus.cc:2258] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:03.834300 31430 ts_tablet_manager.cc:1905] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:03.834367 31429 ts_tablet_manager.cc:1905] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:03.836027 31428 ts_tablet_manager.cc:1905] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:03.842636 31429 ts_tablet_manager.cc:1918] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:03.842840 31429 log.cc:1196] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/ab53ef789cf94afaaf9265e975b3554a
I20260927 07:05:03.843194 31430 ts_tablet_manager.cc:1918] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:03.843497 31429 ts_tablet_manager.cc:1939] T ab53ef789cf94afaaf9265e975b3554a P 388dba9cd7994569b561dc6683e93afe: Deleting consensus metadata
I20260927 07:05:03.843683 31430 log.cc:1196] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/ab53ef789cf94afaaf9265e975b3554a
I20260927 07:05:03.844291 31430 ts_tablet_manager.cc:1939] T ab53ef789cf94afaaf9265e975b3554a P e0f6f8e2af9749f780da9a93b6f6f052: Deleting consensus metadata
I20260927 07:05:03.844805 31428 ts_tablet_manager.cc:1918] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:03.844993 31428 log.cc:1196] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/ab53ef789cf94afaaf9265e975b3554a
I20260927 07:05:03.845511 31428 ts_tablet_manager.cc:1939] T ab53ef789cf94afaaf9265e975b3554a P 94cf0340b8af4e6185007d53d4a4a6eb: Deleting consensus metadata
I20260927 07:05:03.845623 30915 catalog_manager.cc:4841] TS 388dba9cd7994569b561dc6683e93afe (127.13.121.193:35793): tablet ab53ef789cf94afaaf9265e975b3554a (table test-table [id=36c1dabd58cd4e3ca8e21c1f6c30049d]) successfully deleted
I20260927 07:05:03.846602 30914 catalog_manager.cc:4841] TS e0f6f8e2af9749f780da9a93b6f6f052 (127.13.121.195:34473): tablet ab53ef789cf94afaaf9265e975b3554a (table test-table [id=36c1dabd58cd4e3ca8e21c1f6c30049d]) successfully deleted
I20260927 07:05:03.847177 30915 catalog_manager.cc:4841] TS 94cf0340b8af4e6185007d53d4a4a6eb (127.13.121.194:35917): tablet ab53ef789cf94afaaf9265e975b3554a (table test-table [id=36c1dabd58cd4e3ca8e21c1f6c30049d]) successfully deleted
W20260927 07:05:03.848274 30915 catalog_manager.cc:4808] TS 388dba9cd7994569b561dc6683e93afe (127.13.121.193:35793): delete failed for tablet ab53ef789cf94afaaf9265e975b3554a because the tablet was not found. No further retry: Not found: Tablet not found: ab53ef789cf94afaaf9265e975b3554a
Sep 27 07:05:03 dist-test-slave-p6hz krb5kdc[30883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:05:03.856668 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.818523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.13.121.193:35793)
0927 07:05:03.818712 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.818718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.828938 (+ 10220us) server_negotiation.cc:407] Connection header received
0927 07:05:03.829129 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.829136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.829230 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.829327 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:03.831263 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.832442 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.835732 (+ 3290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:03.836392 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:03.836501 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:03.850443 (+ 13942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:03.850457 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:03.850472 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:03.850513 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:03.853348 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:03.854319 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:03.854327 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:03.854330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:03.854477 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:03.854912 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:03.854916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:03.854918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:03.855423 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:03.855668 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:03.856122 (+ 454us) server_negotiation.cc:299] Negotiation successful
0927 07:05:03.856311 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:05:03.857292 31112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:03.981602 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 30989
I20260927 07:05:03.996310 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 31123
Sep 27 07:05:04 dist-test-slave-p6hz krb5kdc[30883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:05:04.014246 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:03.997136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39826 (local address 127.13.121.194:35917)
0927 07:05:03.997292 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:03.997296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:03.997314 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:05:03.997404 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:03.997408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:03.997482 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:03.997561 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:03.999189 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:04.000037 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:04.001934 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:04.002425 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:04.002534 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:04.008001 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:04.008019 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:04.008035 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:04.008093 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:04.011104 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:04.012008 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:04.012015 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:04.012018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:04.012194 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:04.012622 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:04.012627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:04.012629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:04.013010 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:04.013300 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:04.013718 (+ 418us) server_negotiation.cc:299] Negotiation successful
0927 07:05:04.013907 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 07:05:04.015027 31246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:04.130683 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 31123
I20260927 07:05:04.148252 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 31257
Sep 27 07:05:04 dist-test-slave-p6hz krb5kdc[30883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:05:04.163672 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:04.149014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.13.121.195:34473)
0927 07:05:04.149241 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:04.149246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:04.149265 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:05:04.149321 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:04.149324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:04.149385 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:04.149490 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:04.150766 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:04.151629 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:04.152973 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:04.153414 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:04.153526 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:04.158069 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:04.158088 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:04.158104 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:04.158149 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:04.160857 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:04.161684 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:04.161692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:04.161694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:04.161840 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:04.162158 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:04.162162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:04.162165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:04.162554 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:04.162799 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:04.163121 (+ 322us) server_negotiation.cc:299] Negotiation successful
0927 07:05:04.163349 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260927 07:05:04.164350 31380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:04.253296 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 31257
I20260927 07:05:04.266525 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 30896
I20260927 07:05:04.267663 30958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:04.380357 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 30896
2026-09-27T07:05:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4282 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-27T07:05:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:04Z Disabled control of system clock
I20260927 07:05:04.456305 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41031
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:05:04.794699 31439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:04.795027 31439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:04.795117 31439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:04.802562 31439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:04.802690 31439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:04.802726 31439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:04.802774 31439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:04.802817 31439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:04.812294 31439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:04.813588 31439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:04.815439 31439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:04.827811 31444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:04.827783 31445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:04.827756 31447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:04.828289 31439 server_base.cc:1034] running on GCE node
I20260927 07:05:04.829248 31439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:04.830744 31439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:04.831990 31439 hybrid_clock.cc:648] HybridClock initialized: now 1790492704831927 us; error 46 us; skew 500 ppm
I20260927 07:05:04.832396 31439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:04.840514 31439 webserver.cc:460] Webserver started at http://127.13.121.254:41323/ using document root <none> and password file <none>
I20260927 07:05:04.841331 31439 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:04.841447 31439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:04.841763 31439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:04.844905 31439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69"
format_stamp: "Formatted at 2026-09-27 07:05:04 on dist-test-slave-p6hz"
server_key: "8723af99ff7648982f8467d278f47db2"
server_key_iv: "91117d82f639822dd9c65a180b2e63e3"
server_key_version: "encryptionkey@0"
I20260927 07:05:04.845782 31439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69"
format_stamp: "Formatted at 2026-09-27 07:05:04 on dist-test-slave-p6hz"
server_key: "8723af99ff7648982f8467d278f47db2"
server_key_iv: "91117d82f639822dd9c65a180b2e63e3"
server_key_version: "encryptionkey@0"
I20260927 07:05:04.851089 31439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 07:05:04.854955 31453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:04.856505 31439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:04.856709 31439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69"
format_stamp: "Formatted at 2026-09-27 07:05:04 on dist-test-slave-p6hz"
server_key: "8723af99ff7648982f8467d278f47db2"
server_key_iv: "91117d82f639822dd9c65a180b2e63e3"
server_key_version: "encryptionkey@0"
I20260927 07:05:04.856931 31439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:04.877782 31439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:04.878667 31439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:04.878947 31439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:04.896883 31439 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:41031
I20260927 07:05:04.896888 31504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:41031 every 8 connection(s)
I20260927 07:05:04.898234 31439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:04.899837 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 31439
I20260927 07:05:04.900015 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:04.900575 13799 external_mini_cluster.cc:1428] Setting key ad0985b3d55c62b205ae4df852de5798
I20260927 07:05:04.902798 31505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:04.914254 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:04.902468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.13.121.254:41031)
0927 07:05:04.903498 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:04.903509 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:04.903552 (+ 43us) server_negotiation.cc:407] Connection header received
0927 07:05:04.905167 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:04.905188 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:04.905663 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:04.906006 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:04.906606 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:04.908007 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:04.909912 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:04.910588 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:04.910719 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:04.911912 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:04.911922 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:04.911937 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:04.912506 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:04.912589 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:04.912612 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:04.912697 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:04.912924 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 07:05:04.913287 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":341,"threads_started":1}
I20260927 07:05:04.915489 31505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69: Bootstrap starting.
I20260927 07:05:04.918740 31505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:04.919793 31505 log.cc:824] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:04.922797 31505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69: No bootstrap required, opened a new log
I20260927 07:05:04.930459 31505 raft_consensus.cc:348] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41031 } }
I20260927 07:05:04.930862 31505 raft_consensus.cc:374] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:04.930959 31505 raft_consensus.cc:729] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96b23f7e3134c3ab3ad607bbe4a1e69, State: Initialized, Role: FOLLOWER
I20260927 07:05:04.931555 31505 consensus_queue.cc:260] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41031 } }
I20260927 07:05:04.931782 31505 raft_consensus.cc:388] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:04.931916 31505 raft_consensus.cc:482] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:04.932073 31505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:04.934749 31505 raft_consensus.cc:504] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41031 } }
I20260927 07:05:04.935410 31505 leader_election.cc:302] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e96b23f7e3134c3ab3ad607bbe4a1e69; no voters:
I20260927 07:05:04.936020 31505 leader_election.cc:288] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:04.936193 31510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:04.936885 31510 raft_consensus.cc:686] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [term 1 LEADER]: Becoming Leader. State: Replica: e96b23f7e3134c3ab3ad607bbe4a1e69, State: Running, Role: LEADER
I20260927 07:05:04.937430 31510 consensus_queue.cc:237] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41031 } }
I20260927 07:05:04.938236 31505 sys_catalog.cc:564] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:04.940287 31512 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e96b23f7e3134c3ab3ad607bbe4a1e69. Latest consensus state: current_term: 1 leader_uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41031 } } }
I20260927 07:05:04.940763 31512 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:04.940649 31511 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96b23f7e3134c3ab3ad607bbe4a1e69" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41031 } } }
I20260927 07:05:04.942313 31511 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:04.942624 31518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:04.948562 31518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:04.957671 31518 catalog_manager.cc:1347] Generated new cluster ID: 39796bf79edd4d5ba335f7dc8c05241d
I20260927 07:05:04.957809 31518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:04.969077 31518 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:04.970397 31518 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:04.977613 31518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69: Generated new TSK 0
I20260927 07:05:04.978473 31518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:04.993225 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41031
--builtin_ntp_servers=127.13.121.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:05:05.323576 31529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:05.323897 31529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:05.324028 31529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:05.331555 31529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:05.331682 31529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:05.331835 31529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:05:05.341420 31529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:05.342760 31529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:05.344887 31529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:05.358295 31537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:05.358245 31535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:05.358260 31534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:05.359040 31529 server_base.cc:1034] running on GCE node
I20260927 07:05:05.359690 31529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:05.360739 31529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:05.361969 31529 hybrid_clock.cc:648] HybridClock initialized: now 1790492705361918 us; error 39 us; skew 500 ppm
I20260927 07:05:05.362306 31529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:05.364636 31529 webserver.cc:460] Webserver started at http://127.13.121.193:44417/ using document root <none> and password file <none>
I20260927 07:05:05.365320 31529 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:05.365415 31529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:05.365721 31529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:05.368716 31529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "9d2fb66444404d3f941cae7799f804ce"
format_stamp: "Formatted at 2026-09-27 07:05:05 on dist-test-slave-p6hz"
server_key: "1d9ee5d2b950ed312353d1c291a32f88"
server_key_iv: "86900e01e10877f5690a49df0870699e"
server_key_version: "encryptionkey@0"
I20260927 07:05:05.369508 31529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "9d2fb66444404d3f941cae7799f804ce"
format_stamp: "Formatted at 2026-09-27 07:05:05 on dist-test-slave-p6hz"
server_key: "1d9ee5d2b950ed312353d1c291a32f88"
server_key_iv: "86900e01e10877f5690a49df0870699e"
server_key_version: "encryptionkey@0"
I20260927 07:05:05.374771 31529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:05:05.378327 31543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:05.379750 31529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:05.379911 31529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "9d2fb66444404d3f941cae7799f804ce"
format_stamp: "Formatted at 2026-09-27 07:05:05 on dist-test-slave-p6hz"
server_key: "1d9ee5d2b950ed312353d1c291a32f88"
server_key_iv: "86900e01e10877f5690a49df0870699e"
server_key_version: "encryptionkey@0"
I20260927 07:05:05.380118 31529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:05.408607 31529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:05.409490 31529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:05.409781 31529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:05.410759 31529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:05.412271 31529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:05.412364 31529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:05.412477 31529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:05.412544 31529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:05.442905 31529 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:42005
I20260927 07:05:05.442968 31655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:42005 every 8 connection(s)
I20260927 07:05:05.444325 31529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:05:05.446208 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 31529
I20260927 07:05:05.446385 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:05:05.446806 13799 external_mini_cluster.cc:1428] Setting key 37b4cff8937ac71b0979fbe8bb8905a2
I20260927 07:05:05.449571 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41031
--builtin_ntp_servers=127.13.121.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:05:05.458811 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:05.447019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:47917 (local address 127.13.121.254:41031)
0927 07:05:05.447530 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:05.447537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:05.448997 (+ 1460us) server_negotiation.cc:407] Connection header received
0927 07:05:05.450963 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:05.450970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:05.451057 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:05.451197 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:05.452426 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:05.453506 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:05.455371 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:05.455956 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:05.456102 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:05.457863 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:05.457869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:05.457872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:05.457896 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:05.457938 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:05.457951 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:05.458030 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:05.458370 (+ 340us) server_negotiation.cc:299] Negotiation successful
0927 07:05:05.458493 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":252,"threads_started":1}
I20260927 07:05:05.460330 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:05.447737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41031 (local address 127.13.121.193:47917)
0927 07:05:05.448830 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:05.448877 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:05.450665 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:05.451465 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:05.451475 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:05.451597 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:05.452233 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:05.452259 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:05.453725 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:05.453729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:05.455094 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:05.455112 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:05.456263 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:05.456270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:05.456529 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:05.457404 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:05:05.457408 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:05.457482 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:05:05.457608 (+ 126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:05.458061 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:05.458074 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:05.458082 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:05:05.458281 (+ 199us) client_negotiation.cc:240] Negotiation successful
0927 07:05:05.458754 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":318,"threads_started":1}
I20260927 07:05:05.462047 31656 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41031
I20260927 07:05:05.462363 31656 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:05.463224 31656 heartbeater.cc:511] Master 127.13.121.254:41031 requested a full tablet report, sending...
I20260927 07:05:05.465533 31470 ts_manager.cc:194] Registered new tserver with Master: 9d2fb66444404d3f941cae7799f804ce (127.13.121.193:42005)
I20260927 07:05:05.467684 31470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.193:47917
W20260927 07:05:05.781670 31660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:05.781973 31660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:05.782058 31660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:05.789623 31660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:05.789748 31660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:05.789861 31660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:05:05.799386 31660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:05.800666 31660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:05.802453 31660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:05.815307 31665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:05.815397 31668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:05.815400 31666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:05.816435 31660 server_base.cc:1034] running on GCE node
I20260927 07:05:05.817049 31660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:05.818048 31660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:05.819240 31660 hybrid_clock.cc:648] HybridClock initialized: now 1790492705819189 us; error 49 us; skew 500 ppm
I20260927 07:05:05.819563 31660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:05.822046 31660 webserver.cc:460] Webserver started at http://127.13.121.194:46819/ using document root <none> and password file <none>
I20260927 07:05:05.822777 31660 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:05.822883 31660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:05.823258 31660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:05.826162 31660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "1b546a4bec98440ab360946c873b465c"
format_stamp: "Formatted at 2026-09-27 07:05:05 on dist-test-slave-p6hz"
server_key: "f189e2edaa8e29317cc3ba94240523de"
server_key_iv: "4df58242ec4a2a73450fd74d3405b9e8"
server_key_version: "encryptionkey@0"
I20260927 07:05:05.826941 31660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "1b546a4bec98440ab360946c873b465c"
format_stamp: "Formatted at 2026-09-27 07:05:05 on dist-test-slave-p6hz"
server_key: "f189e2edaa8e29317cc3ba94240523de"
server_key_iv: "4df58242ec4a2a73450fd74d3405b9e8"
server_key_version: "encryptionkey@0"
I20260927 07:05:05.832098 31660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260927 07:05:05.836043 31674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:05.837440 31660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:05.837616 31660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "1b546a4bec98440ab360946c873b465c"
format_stamp: "Formatted at 2026-09-27 07:05:05 on dist-test-slave-p6hz"
server_key: "f189e2edaa8e29317cc3ba94240523de"
server_key_iv: "4df58242ec4a2a73450fd74d3405b9e8"
server_key_version: "encryptionkey@0"
I20260927 07:05:05.837827 31660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:05.858707 31660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:05.859685 31660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:05.860016 31660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:05.861070 31660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:05.862560 31660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:05.862654 31660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:05.862761 31660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:05.862828 31660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:05.893335 31660 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:38045
I20260927 07:05:05.893405 31786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:38045 every 8 connection(s)
I20260927 07:05:05.894594 31660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:05:05.896229 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 31660
I20260927 07:05:05.896404 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:05:05.896950 13799 external_mini_cluster.cc:1428] Setting key dba3c8c780a4031b56e990be0e2f09f4
I20260927 07:05:05.900132 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41031
--builtin_ntp_servers=127.13.121.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:05:05.907567 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:05.897251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:55127 (local address 127.13.121.254:41031)
0927 07:05:05.897449 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:05.897455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:05.899242 (+ 1787us) server_negotiation.cc:407] Connection header received
0927 07:05:05.901022 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:05.901028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:05.901121 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:05.901204 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:05.902376 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:05.903224 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:05.904809 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:05.905240 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:05.905350 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:05.906742 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:05.906747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:05.906750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:05.906769 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:05.906801 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:05.906812 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:05.906882 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:05.907104 (+ 222us) server_negotiation.cc:299] Negotiation successful
0927 07:05:05.907240 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 07:05:05.908748 31789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:05.897832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41031 (local address 127.13.121.194:55127)
0927 07:05:05.899087 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:05.899122 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:05.900752 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:05.901478 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:05.901489 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:05.901624 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:05.902198 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:05.902216 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:05.903378 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:05.903383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:05.904665 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:05.904679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:05.905466 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:05.905474 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:05.905719 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:05.906407 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:05:05.906409 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:05.906451 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:05:05.906540 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:05.906899 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:05.906910 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:05.906918 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:05:05.907074 (+ 156us) client_negotiation.cc:240] Negotiation successful
0927 07:05:05.907527 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":321,"threads_started":1}
I20260927 07:05:05.910251 31787 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41031
I20260927 07:05:05.910604 31787 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:05.911561 31787 heartbeater.cc:511] Master 127.13.121.254:41031 requested a full tablet report, sending...
I20260927 07:05:05.913194 31470 ts_manager.cc:194] Registered new tserver with Master: 1b546a4bec98440ab360946c873b465c (127.13.121.194:38045)
I20260927 07:05:05.914074 31470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.194:55127
W20260927 07:05:06.232088 31790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:06.232391 31790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:06.232501 31790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:06.240442 31790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:06.240561 31790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:06.240700 31790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:05:06.250061 31790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:06.251362 31790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:06.253060 31790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:06.266669 31795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:06.266697 31796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:06.266741 31798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:06.267266 31790 server_base.cc:1034] running on GCE node
I20260927 07:05:06.267916 31790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:06.268855 31790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:06.270056 31790 hybrid_clock.cc:648] HybridClock initialized: now 1790492706270003 us; error 39 us; skew 500 ppm
I20260927 07:05:06.270357 31790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:06.272567 31790 webserver.cc:460] Webserver started at http://127.13.121.195:38999/ using document root <none> and password file <none>
I20260927 07:05:06.273252 31790 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:06.273353 31790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:06.273648 31790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:06.276518 31790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "062ed17c2baf48eb9c6c6de20b712b9e"
format_stamp: "Formatted at 2026-09-27 07:05:06 on dist-test-slave-p6hz"
server_key: "448ca8d79c4bbfc52a67b08ac8398fab"
server_key_iv: "ca4ec14b9a1a6cc05d27865f4f7a07f7"
server_key_version: "encryptionkey@0"
I20260927 07:05:06.277272 31790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "062ed17c2baf48eb9c6c6de20b712b9e"
format_stamp: "Formatted at 2026-09-27 07:05:06 on dist-test-slave-p6hz"
server_key: "448ca8d79c4bbfc52a67b08ac8398fab"
server_key_iv: "ca4ec14b9a1a6cc05d27865f4f7a07f7"
server_key_version: "encryptionkey@0"
I20260927 07:05:06.282342 31790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260927 07:05:06.285948 31804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:06.287376 31790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:06.287533 31790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "062ed17c2baf48eb9c6c6de20b712b9e"
format_stamp: "Formatted at 2026-09-27 07:05:06 on dist-test-slave-p6hz"
server_key: "448ca8d79c4bbfc52a67b08ac8398fab"
server_key_iv: "ca4ec14b9a1a6cc05d27865f4f7a07f7"
server_key_version: "encryptionkey@0"
I20260927 07:05:06.287725 31790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:06.305502 31790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:06.306367 31790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:06.306651 31790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:06.307693 31790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:06.309229 31790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:06.309330 31790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:06.309438 31790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:06.309505 31790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:06.340211 31790 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:39315
I20260927 07:05:06.340265 31916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:39315 every 8 connection(s)
I20260927 07:05:06.341557 31790 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:05:06.349198 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 31790
I20260927 07:05:06.349386 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:05:06.349932 13799 external_mini_cluster.cc:1428] Setting key 6ea682fdb66195ef004d9aa0e213a581
I20260927 07:05:06.354739 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.344178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:54033 (local address 127.13.121.254:41031)
0927 07:05:06.344394 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.344400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.345809 (+ 1409us) server_negotiation.cc:407] Connection header received
0927 07:05:06.347679 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.347684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.347760 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.347853 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:06.348992 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.350084 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.352103 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.352570 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.352687 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.354012 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:06.354017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:06.354020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:06.354039 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:06.354069 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:06.354080 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:06.354147 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.354410 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.354492 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:05:06.355809 31919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.344740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41031 (local address 127.13.121.195:54033)
0927 07:05:06.345654 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.345692 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:06.347377 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:06.348054 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:06.348062 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:06.348188 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:06.348790 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.348810 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.350235 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.350239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.351953 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.351971 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.352796 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.352804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.353033 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.353711 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:05:06.353714 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:06.353757 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:05:06.353842 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:06.354208 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:06.354218 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:06.354227 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 07:05:06.354403 (+ 176us) client_negotiation.cc:240] Negotiation successful
0927 07:05:06.354730 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":275,"threads_started":1}
I20260927 07:05:06.357115 31917 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41031
I20260927 07:05:06.357400 31917 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:06.358145 31917 heartbeater.cc:511] Master 127.13.121.254:41031 requested a full tablet report, sending...
I20260927 07:05:06.359544 31470 ts_manager.cc:194] Registered new tserver with Master: 062ed17c2baf48eb9c6c6de20b712b9e (127.13.121.195:39315)
I20260927 07:05:06.360360 31470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.195:54033
I20260927 07:05:06.365031 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:05:06.374665 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.368959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57924 (local address 127.13.121.254:41031)
0927 07:05:06.369179 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.369185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.369470 (+ 285us) server_negotiation.cc:407] Connection header received
0927 07:05:06.369647 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.369653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.369727 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.369810 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:06.370297 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.370992 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.372423 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.372842 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.372955 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.373991 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:06.373997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:06.374000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:06.374019 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:06.374053 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:06.374065 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:06.374139 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.374267 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.374376 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 07:05:06.378711 31470 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:05:06.385587 31470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:05:06.411294 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.398583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.13.121.193:42005)
0927 07:05:06.400599 (+ 2016us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.400609 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.400643 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:05:06.400759 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.400768 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.401038 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.402445 (+ 1407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:06.402710 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.404462 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.408203 (+ 3741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.409707 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.409867 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.410089 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.410710 (+ 621us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.410898 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1841,"thread_start_us":1712,"threads_started":1}
I20260927 07:05:06.411814 31929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.398161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.0.0.1:43726)
0927 07:05:06.399725 (+ 1564us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.399779 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:06.399978 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:06.401699 (+ 1721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:06.401708 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:06.401745 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:06.402214 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.402227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.405594 (+ 3367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.405602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.408038 (+ 2436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.408047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.410229 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.410236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.410436 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.410492 (+ 56us) client_negotiation.cc:769] Sending connection context
0927 07:05:06.411217 (+ 725us) client_negotiation.cc:240] Negotiation successful
0927 07:05:06.411419 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1264,"mutex_wait_us":1,"thread_start_us":1086,"threads_started":1}
I20260927 07:05:06.413290 31930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.398285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:38045 (local address 127.0.0.1:55546)
0927 07:05:06.400920 (+ 2635us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.400939 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:06.401056 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:06.405989 (+ 4933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:06.405994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:06.406030 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:06.406491 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.406504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.408330 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.408335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.410339 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.410353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.412486 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.412493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.412673 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.412705 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:05:06.412814 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 07:05:06.412952 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2496,"mutex_wait_us":4,"thread_start_us":2868,"threads_started":1}
I20260927 07:05:06.413950 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.398656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55546 (local address 127.13.121.194:38045)
0927 07:05:06.405035 (+ 6379us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.405045 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.405078 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:05:06.405199 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.405208 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.405506 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.405715 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:06.406764 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.408187 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.410578 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.412282 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.412444 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.412651 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.412859 (+ 208us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.413032 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6205,"thread_start_us":6076,"threads_started":1}
I20260927 07:05:06.414640 31933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.401169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.13.121.195:39315)
0927 07:05:06.405415 (+ 4246us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.405422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.406156 (+ 734us) server_negotiation.cc:407] Connection header received
0927 07:05:06.406480 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.406488 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.406815 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.406992 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:06.407856 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.408959 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.411793 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.413137 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.413353 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.413552 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.413935 (+ 383us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.414290 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4089,"thread_start_us":3940,"threads_started":1}
I20260927 07:05:06.415395 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.400738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:39315 (local address 127.0.0.1:34296)
0927 07:05:06.406037 (+ 5299us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.406056 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:06.406262 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:06.407096 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:06.407101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:06.407126 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:06.407650 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.407662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.409092 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.409096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.411573 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.411589 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.413535 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.413546 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.413724 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.413757 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 07:05:06.414975 (+ 1218us) client_negotiation.cc:240] Negotiation successful
0927 07:05:06.415116 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5159,"mutex_wait_us":633,"thread_start_us":4347,"threads_started":1}
I20260927 07:05:06.416234 31591 tablet_service.cc:1467] Processing CreateTablet for tablet 010055a5460e4614b1cc52a52833447a (DEFAULT_TABLE table=test-table [id=6c109daaa0f54684883582521219136a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:06.417754 31591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010055a5460e4614b1cc52a52833447a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:06.417613 31722 tablet_service.cc:1467] Processing CreateTablet for tablet 010055a5460e4614b1cc52a52833447a (DEFAULT_TABLE table=test-table [id=6c109daaa0f54684883582521219136a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:06.418424 31852 tablet_service.cc:1467] Processing CreateTablet for tablet 010055a5460e4614b1cc52a52833447a (DEFAULT_TABLE table=test-table [id=6c109daaa0f54684883582521219136a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:06.418977 31722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010055a5460e4614b1cc52a52833447a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:06.419597 31852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010055a5460e4614b1cc52a52833447a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:06.431367 31935 tablet_bootstrap.cc:492] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: Bootstrap starting.
I20260927 07:05:06.435186 31935 tablet_bootstrap.cc:654] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:06.435973 31936 tablet_bootstrap.cc:492] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce: Bootstrap starting.
I20260927 07:05:06.436805 31935 log.cc:824] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:06.438539 31937 tablet_bootstrap.cc:492] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: Bootstrap starting.
I20260927 07:05:06.439976 31936 tablet_bootstrap.cc:654] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:06.441244 31935 tablet_bootstrap.cc:492] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: No bootstrap required, opened a new log
I20260927 07:05:06.441308 31936 log.cc:824] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:06.441571 31935 ts_tablet_manager.cc:1397] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260927 07:05:06.445179 31936 tablet_bootstrap.cc:492] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce: No bootstrap required, opened a new log
I20260927 07:05:06.445472 31936 ts_tablet_manager.cc:1397] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260927 07:05:06.445964 31937 tablet_bootstrap.cc:654] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:06.447309 31937 log.cc:824] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:06.453729 31935 raft_consensus.cc:348] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } }
I20260927 07:05:06.454366 31935 raft_consensus.cc:374] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:06.454480 31935 raft_consensus.cc:729] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 062ed17c2baf48eb9c6c6de20b712b9e, State: Initialized, Role: FOLLOWER
I20260927 07:05:06.455526 31935 consensus_queue.cc:260] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } }
I20260927 07:05:06.457125 31917 heartbeater.cc:503] Master 127.13.121.254:41031 was elected leader, sending a full tablet report...
I20260927 07:05:06.456974 31936 raft_consensus.cc:348] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } }
I20260927 07:05:06.457494 31936 raft_consensus.cc:374] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:06.457597 31936 raft_consensus.cc:729] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d2fb66444404d3f941cae7799f804ce, State: Initialized, Role: FOLLOWER
I20260927 07:05:06.458182 31935 ts_tablet_manager.cc:1428] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260927 07:05:06.458336 31936 consensus_queue.cc:260] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } }
I20260927 07:05:06.460889 31937 tablet_bootstrap.cc:492] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: No bootstrap required, opened a new log
I20260927 07:05:06.461242 31937 ts_tablet_manager.cc:1397] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.008s
I20260927 07:05:06.463351 31656 heartbeater.cc:503] Master 127.13.121.254:41031 was elected leader, sending a full tablet report...
I20260927 07:05:06.463943 31936 ts_tablet_manager.cc:1428] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
I20260927 07:05:06.470935 31937 raft_consensus.cc:348] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } }
I20260927 07:05:06.471418 31937 raft_consensus.cc:374] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:06.471522 31937 raft_consensus.cc:729] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b546a4bec98440ab360946c873b465c, State: Initialized, Role: FOLLOWER
I20260927 07:05:06.472301 31937 consensus_queue.cc:260] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } }
I20260927 07:05:06.473677 31787 heartbeater.cc:503] Master 127.13.121.254:41031 was elected leader, sending a full tablet report...
I20260927 07:05:06.474210 31937 ts_tablet_manager.cc:1428] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260927 07:05:06.553887 31941 raft_consensus.cc:482] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:05:06.554224 31941 raft_consensus.cc:504] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } }
I20260927 07:05:06.556147 31941 leader_election.cc:288] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d2fb66444404d3f941cae7799f804ce (127.13.121.193:42005), 1b546a4bec98440ab360946c873b465c (127.13.121.194:38045)
I20260927 07:05:06.562937 31919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.556753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.13.121.195:43111)
0927 07:05:06.556982 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.557008 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:06.557207 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:06.557708 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:06.557713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:06.557746 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:06.557937 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.557949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.559211 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.559217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.560864 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.560876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.562156 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.562162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.562318 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.562341 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 07:05:06.562437 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 07:05:06.562592 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":1}
I20260927 07:05:06.563086 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.556751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:57857 (local address 127.13.121.194:38045)
0927 07:05:06.557121 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.557127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.557662 (+ 535us) server_negotiation.cc:407] Connection header received
0927 07:05:06.557800 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.557805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.557879 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.557984 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:06.558624 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.559532 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.561413 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.562227 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.562369 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.562437 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.562690 (+ 253us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.562816 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260927 07:05:06.563468 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.556835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:43111 (local address 127.13.121.193:42005)
0927 07:05:06.557038 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.557043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.557189 (+ 146us) server_negotiation.cc:407] Connection header received
0927 07:05:06.557361 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.557367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.557466 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.557578 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:06.558150 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.559004 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.561032 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.562001 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.562156 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.562229 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.563053 (+ 824us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.563224 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 07:05:06.563917 31944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.556811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:38045 (local address 127.13.121.195:57857)
0927 07:05:06.557529 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.557547 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:06.557679 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:06.558028 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:06.558033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:06.558059 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:06.558434 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.558446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.559700 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.559703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.561224 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:06.561237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.562384 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.562390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:06.562541 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.562565 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 07:05:06.562649 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 07:05:06.562784 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":372,"threads_started":1}
I20260927 07:05:06.564491 31611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010055a5460e4614b1cc52a52833447a" candidate_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d2fb66444404d3f941cae7799f804ce" is_pre_election: true
I20260927 07:05:06.564910 31742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010055a5460e4614b1cc52a52833447a" candidate_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b546a4bec98440ab360946c873b465c" is_pre_election: true
I20260927 07:05:06.565006 31611 raft_consensus.cc:2454] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 062ed17c2baf48eb9c6c6de20b712b9e in term 0.
I20260927 07:05:06.565371 31742 raft_consensus.cc:2454] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 062ed17c2baf48eb9c6c6de20b712b9e in term 0.
I20260927 07:05:06.565807 31808 leader_election.cc:302] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 062ed17c2baf48eb9c6c6de20b712b9e, 9d2fb66444404d3f941cae7799f804ce; no voters:
I20260927 07:05:06.566259 31941 raft_consensus.cc:2781] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:05:06.566433 31941 raft_consensus.cc:482] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:05:06.566551 31941 raft_consensus.cc:3037] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:06.568826 31941 raft_consensus.cc:504] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } }
I20260927 07:05:06.569489 31941 leader_election.cc:288] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [CANDIDATE]: Term 1 election: Requested vote from peers 9d2fb66444404d3f941cae7799f804ce (127.13.121.193:42005), 1b546a4bec98440ab360946c873b465c (127.13.121.194:38045)
I20260927 07:05:06.570040 31611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010055a5460e4614b1cc52a52833447a" candidate_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d2fb66444404d3f941cae7799f804ce"
I20260927 07:05:06.570057 31742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010055a5460e4614b1cc52a52833447a" candidate_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b546a4bec98440ab360946c873b465c"
I20260927 07:05:06.570247 31611 raft_consensus.cc:3037] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:06.570278 31742 raft_consensus.cc:3037] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:06.572559 31611 raft_consensus.cc:2454] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 062ed17c2baf48eb9c6c6de20b712b9e in term 1.
I20260927 07:05:06.572841 31742 raft_consensus.cc:2454] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 062ed17c2baf48eb9c6c6de20b712b9e in term 1.
I20260927 07:05:06.573021 31808 leader_election.cc:302] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 062ed17c2baf48eb9c6c6de20b712b9e, 9d2fb66444404d3f941cae7799f804ce; no voters:
I20260927 07:05:06.573385 31941 raft_consensus.cc:2781] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:06.573982 31941 raft_consensus.cc:686] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 1 LEADER]: Becoming Leader. State: Replica: 062ed17c2baf48eb9c6c6de20b712b9e, State: Running, Role: LEADER
I20260927 07:05:06.575049 31941 consensus_queue.cc:237] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } }
I20260927 07:05:06.580068 31469 catalog_manager.cc:5495] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 062ed17c2baf48eb9c6c6de20b712b9e (127.13.121.195). New cstate: current_term: 1 leader_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "062ed17c2baf48eb9c6c6de20b712b9e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 39315 } health_report { overall_health: HEALTHY } } }
W20260927 07:05:06.593194 31918 tablet.cc:2369] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:05:06.612689 31933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.607609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.13.121.195:39315)
0927 07:05:06.607774 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.607779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.607797 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:05:06.607959 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.607963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.608052 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.608188 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:06.608459 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.609188 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.610593 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.611032 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.611165 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.611662 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:06.612197 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:06.612305 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.612363 (+ 58us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.612433 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 07:05:06.621187 31742 raft_consensus.cc:1261] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [term 1 FOLLOWER]: Refusing update from remote peer 062ed17c2baf48eb9c6c6de20b712b9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:06.621413 31611 raft_consensus.cc:1261] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [term 1 FOLLOWER]: Refusing update from remote peer 062ed17c2baf48eb9c6c6de20b712b9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:06.622246 31945 consensus_queue.cc:1035] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b546a4bec98440ab360946c873b465c" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:06.622611 31941 consensus_queue.cc:1035] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d2fb66444404d3f941cae7799f804ce" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:06.637921 31950 mvcc.cc:205] Tried to move back new op lower bound from 7333858126309937152 to 7333858126136553472. Current Snapshot: MvccSnapshot[applied={T|T < 7333858126309937152}]
I20260927 07:05:06.649838 31470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:06.650115 31470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:06.654316 31470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e96b23f7e3134c3ab3ad607bbe4a1e69: Sending DeleteTablet for 3 replicas of tablet 010055a5460e4614b1cc52a52833447a
I20260927 07:05:06.655701 31852 tablet_service.cc:1514] Processing DeleteTablet for tablet 010055a5460e4614b1cc52a52833447a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:06 UTC) from {username='slave'} at 127.0.0.1:34296
I20260927 07:05:06.655671 31591 tablet_service.cc:1514] Processing DeleteTablet for tablet 010055a5460e4614b1cc52a52833447a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:06 UTC) from {username='slave'} at 127.0.0.1:43726
I20260927 07:05:06.655720 31722 tablet_service.cc:1514] Processing DeleteTablet for tablet 010055a5460e4614b1cc52a52833447a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:06 UTC) from {username='slave'} at 127.0.0.1:55546
I20260927 07:05:06.658732 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 31529
I20260927 07:05:06.669023 31959 tablet_replica.cc:330] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: stopping tablet replica
I20260927 07:05:06.669183 31958 tablet_replica.cc:330] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: stopping tablet replica
I20260927 07:05:06.669708 31958 raft_consensus.cc:2229] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:05:06.670379 31959 raft_consensus.cc:2229] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:06.670469 31958 raft_consensus.cc:2258] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:06.672048 31959 raft_consensus.cc:2258] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:06.672927 31958 ts_tablet_manager.cc:1905] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:06.675086 31959 ts_tablet_manager.cc:1905] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:06.677691 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.660430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.13.121.193:42005)
0927 07:05:06.660638 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.660645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.664485 (+ 3840us) server_negotiation.cc:407] Connection header received
0927 07:05:06.664691 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.664698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.664854 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.664977 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:06.669581 (+ 4604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.670835 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.673166 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.673896 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.674018 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.675702 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:06.675717 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:06.675737 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:06.676552 (+ 815us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:06.676630 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:06.676691 (+ 61us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:06.676803 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.677151 (+ 348us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.677304 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:05:06.679062 31957 tablet_replica.cc:330] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce: stopping tablet replica
I20260927 07:05:06.679183 31651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:06.682538 31958 ts_tablet_manager.cc:1918] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:06.683216 31958 log.cc:1196] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/010055a5460e4614b1cc52a52833447a
I20260927 07:05:06.683918 31958 ts_tablet_manager.cc:1939] T 010055a5460e4614b1cc52a52833447a P 062ed17c2baf48eb9c6c6de20b712b9e: Deleting consensus metadata
I20260927 07:05:06.684228 31959 ts_tablet_manager.cc:1918] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:06.684418 31959 log.cc:1196] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/010055a5460e4614b1cc52a52833447a
I20260927 07:05:06.685166 31959 ts_tablet_manager.cc:1939] T 010055a5460e4614b1cc52a52833447a P 1b546a4bec98440ab360946c873b465c: Deleting consensus metadata
I20260927 07:05:06.686182 31456 catalog_manager.cc:4841] TS 062ed17c2baf48eb9c6c6de20b712b9e (127.13.121.195:39315): tablet 010055a5460e4614b1cc52a52833447a (table test-table [id=6c109daaa0f54684883582521219136a]) successfully deleted
I20260927 07:05:06.687181 31454 catalog_manager.cc:4841] TS 1b546a4bec98440ab360946c873b465c (127.13.121.194:38045): tablet 010055a5460e4614b1cc52a52833447a (table test-table [id=6c109daaa0f54684883582521219136a]) successfully deleted
I20260927 07:05:06.679550 31957 raft_consensus.cc:2229] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:06.778816 31957 raft_consensus.cc:2258] T 010055a5460e4614b1cc52a52833447a P 9d2fb66444404d3f941cae7799f804ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:06.779037 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 31529
W20260927 07:05:06.794756 31457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.121.193:42005 (error 108)
I20260927 07:05:06.794934 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 31660
I20260927 07:05:06.796793 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.796256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.0.0.1:43736)
0927 07:05:06.796448 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.796607 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260927 07:05:06.797071 31457 catalog_manager.cc:4584] TS 9d2fb66444404d3f941cae7799f804ce (127.13.121.193:42005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 010055a5460e4614b1cc52a52833447a: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42005: connect: Connection refused (error 111)
I20260927 07:05:06.802862 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.795792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55548 (local address 127.13.121.194:38045)
0927 07:05:06.795983 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.795989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.796609 (+ 620us) server_negotiation.cc:407] Connection header received
0927 07:05:06.796784 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.796788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.796900 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.797002 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:06.797512 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.798229 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.799620 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.800022 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.800136 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.801254 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:06.801264 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:06.801278 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:06.801850 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:06.801900 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:06.801921 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:06.802030 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.802190 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.802309 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 07:05:06.803618 31782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:06.850848 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.850239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.0.0.1:43752)
0927 07:05:06.850511 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.850629 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260927 07:05:06.852098 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.851695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.0.0.1:43754)
0927 07:05:06.851845 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.851935 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 07:05:06.899534 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 31660
I20260927 07:05:06.912979 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.912411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.0.0.1:43760)
0927 07:05:06.912678 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.912780 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260927 07:05:06.914166 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.913726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.0.0.1:43764)
0927 07:05:06.913933 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.914015 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260927 07:05:06.915158 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 31790
I20260927 07:05:06.922511 31933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.915841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.13.121.195:39315)
0927 07:05:06.916025 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:06.916031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:06.916121 (+ 90us) server_negotiation.cc:407] Connection header received
0927 07:05:06.916313 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:06.916317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:06.916445 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:06.916535 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:06.916965 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.917780 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.919382 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:06.919835 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:06.919961 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:06.921035 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:06.921044 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:06.921059 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:06.921698 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:06.921749 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:06.921771 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:06.921871 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:06.922024 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 07:05:06.922144 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 07:05:06.923403 31912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:06.994982 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.994383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.0.0.1:43772)
0927 07:05:06.994669 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.994793 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260927 07:05:06.996239 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:06.995800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.0.0.1:43780)
0927 07:05:06.996010 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:06.996114 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260927 07:05:07.046604 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 31790
I20260927 07:05:07.062620 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 31439
I20260927 07:05:07.063817 31500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:07.179605 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:07.178706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:42005 (local address 127.0.0.1:43796)
0927 07:05:07.179028 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:07.179236 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260927 07:05:07.179741 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 31439
2026-09-27T07:05:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2799 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:07 dist-test-slave-p6hz krb5kdc[31968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:07 dist-test-slave-p6hz krb5kdc[31968](info): set up 2 sockets
Sep 27 07:05:07 dist-test-slave-p6hz krb5kdc[31968](info): commencing operation
krb5kdc: starting...
W20260927 07:05:08.203250 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:08 dist-test-slave-p6hz krb5kdc[31968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:08.442502 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:32789
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:32789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:08.774084 31981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:08.774392 31981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:08.774528 31981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:08.781912 31981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:08.782037 31981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:08.782100 31981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:08.782181 31981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:08.782233 31981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:08.792357 31981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:32789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:32789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:08.793995 31981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:08.795464 31981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:08.807587 31989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:08.807587 31987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:08.807621 31986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:08.808204 31981 server_base.cc:1034] running on GCE node
I20260927 07:05:08.809209 31981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:08.810725 31981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:08.811934 31981 hybrid_clock.cc:648] HybridClock initialized: now 1790492708811878 us; error 46 us; skew 500 ppm
I20260927 07:05:08.812286 31981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:08 dist-test-slave-p6hz krb5kdc[31968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492708, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:08.815376 31981 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:08.816867 31981 webserver.cc:460] Webserver started at http://127.13.121.254:41407/ using document root <none> and password file <none>
I20260927 07:05:08.817536 31981 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:08.817637 31981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:08.817893 31981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:08.821044 31981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "32eb0c4e762e4341831a3b5f691b5b19"
format_stamp: "Formatted at 2026-09-27 07:05:08 on dist-test-slave-p6hz"
server_key: "92096c031e54263271e1c0f24aae3876"
server_key_iv: "9ba848b3a4bcbb55a627774c4880da98"
server_key_version: "encryptionkey@0"
I20260927 07:05:08.821859 31981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "32eb0c4e762e4341831a3b5f691b5b19"
format_stamp: "Formatted at 2026-09-27 07:05:08 on dist-test-slave-p6hz"
server_key: "92096c031e54263271e1c0f24aae3876"
server_key_iv: "9ba848b3a4bcbb55a627774c4880da98"
server_key_version: "encryptionkey@0"
I20260927 07:05:08.827647 31981 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 07:05:08.831388 31996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:08.832826 31981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:08.832988 31981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "32eb0c4e762e4341831a3b5f691b5b19"
format_stamp: "Formatted at 2026-09-27 07:05:08 on dist-test-slave-p6hz"
server_key: "92096c031e54263271e1c0f24aae3876"
server_key_iv: "9ba848b3a4bcbb55a627774c4880da98"
server_key_version: "encryptionkey@0"
I20260927 07:05:08.833214 31981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:08.853938 31981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:08.858225 31981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:08.858505 31981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:08.876104 31981 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:32789
I20260927 07:05:08.876137 32047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:32789 every 8 connection(s)
I20260927 07:05:08.877482 31981 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:08.881374 32048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:08.885794 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 31981
I20260927 07:05:08.885952 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:08.886348 13799 external_mini_cluster.cc:1428] Setting key b8234629347e0c185bcbead86084125c
I20260927 07:05:08.893968 32048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19: Bootstrap starting.
I20260927 07:05:08.897707 32048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:08.899024 32048 log.cc:824] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:08.902046 32048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19: No bootstrap required, opened a new log
Sep 27 07:05:08 dist-test-slave-p6hz krb5kdc[31968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:08.912241 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:08.888304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52018 (local address 127.13.121.254:32789)
0927 07:05:08.889132 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:08.889149 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:08.889201 (+ 52us) server_negotiation.cc:407] Connection header received
0927 07:05:08.892090 (+ 2889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:08.892125 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:08.892687 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:08.893175 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:08.894616 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:08.896595 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:08.897927 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:08.898967 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:08.899194 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:08.903355 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:08.903366 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:08.903386 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:08.903434 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:08.907460 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:08.908221 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:08.908231 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:08.908245 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:08.908472 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:08.908819 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:08.908823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:08.908826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:08.909597 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:08.909891 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:08.910290 (+ 399us) server_negotiation.cc:299] Negotiation successful
0927 07:05:08.910780 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":354,"threads_started":1}
I20260927 07:05:08.912479 32048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32eb0c4e762e4341831a3b5f691b5b19" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 32789 } }
I20260927 07:05:08.912922 32048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:08.913029 32048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32eb0c4e762e4341831a3b5f691b5b19, State: Initialized, Role: FOLLOWER
I20260927 07:05:08.913648 32048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32eb0c4e762e4341831a3b5f691b5b19" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 32789 } }
I20260927 07:05:08.913887 32048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:08.914013 32048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:08.914182 32048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:08.916548 32048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32eb0c4e762e4341831a3b5f691b5b19" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 32789 } }
I20260927 07:05:08.917048 32048 leader_election.cc:302] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32eb0c4e762e4341831a3b5f691b5b19; no voters:
I20260927 07:05:08.917604 32048 leader_election.cc:288] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:08.917768 32053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:08.918813 32053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [term 1 LEADER]: Becoming Leader. State: Replica: 32eb0c4e762e4341831a3b5f691b5b19, State: Running, Role: LEADER
I20260927 07:05:08.919394 32053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32eb0c4e762e4341831a3b5f691b5b19" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 32789 } }
I20260927 07:05:08.919754 32048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:08.922086 32055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32eb0c4e762e4341831a3b5f691b5b19. Latest consensus state: current_term: 1 leader_uuid: "32eb0c4e762e4341831a3b5f691b5b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32eb0c4e762e4341831a3b5f691b5b19" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 32789 } } }
I20260927 07:05:08.922185 32054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32eb0c4e762e4341831a3b5f691b5b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32eb0c4e762e4341831a3b5f691b5b19" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 32789 } } }
I20260927 07:05:08.922497 32055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:08.922506 32054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:08.923501 32061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:08.930442 32061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:08.941000 32061 catalog_manager.cc:1347] Generated new cluster ID: 0288529c62084ad7b3deb50fe102c42c
I20260927 07:05:08.941139 32061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:08.955732 32061 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:08.957055 32061 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:08.968019 32061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19: Generated new TSK 0
I20260927 07:05:08.968926 32061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:05:09.035089 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:32789
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:09.372709 32074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:09.373003 32074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:09.373134 32074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:09.381170 32074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:09.381319 32074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:09.381479 32074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:05:09.391574 32074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:32789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:09.392931 32074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:09.394260 32074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:09.407449 32079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:09.407500 32080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:09.407655 32082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:09.408627 32074 server_base.cc:1034] running on GCE node
I20260927 07:05:09.409328 32074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:09.410387 32074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:09.411599 32074 hybrid_clock.cc:648] HybridClock initialized: now 1790492709411544 us; error 68 us; skew 500 ppm
I20260927 07:05:09.411918 32074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:09 dist-test-slave-p6hz krb5kdc[31968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492709, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:09.415174 32074 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:05:09.416731 32074 webserver.cc:460] Webserver started at http://127.13.121.193:37501/ using document root <none> and password file <none>
I20260927 07:05:09.417430 32074 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:09.417551 32074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:09.417857 32074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:09.421021 32074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "12e107e2aca2419e81dfd2e7a9624106"
format_stamp: "Formatted at 2026-09-27 07:05:09 on dist-test-slave-p6hz"
server_key: "92448e3332ab01e776c75eeccc7fad35"
server_key_iv: "5983010ba931859d993152e3dc5db142"
server_key_version: "encryptionkey@0"
I20260927 07:05:09.421871 32074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "12e107e2aca2419e81dfd2e7a9624106"
format_stamp: "Formatted at 2026-09-27 07:05:09 on dist-test-slave-p6hz"
server_key: "92448e3332ab01e776c75eeccc7fad35"
server_key_iv: "5983010ba931859d993152e3dc5db142"
server_key_version: "encryptionkey@0"
I20260927 07:05:09.427754 32074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260927 07:05:09.431493 32089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:09.432916 32074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:09.433087 32074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "12e107e2aca2419e81dfd2e7a9624106"
format_stamp: "Formatted at 2026-09-27 07:05:09 on dist-test-slave-p6hz"
server_key: "92448e3332ab01e776c75eeccc7fad35"
server_key_iv: "5983010ba931859d993152e3dc5db142"
server_key_version: "encryptionkey@0"
I20260927 07:05:09.433298 32074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:09.453469 32074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:09.458220 32074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:09.458549 32074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:09.459647 32074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:09.461100 32074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:09.461185 32074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:09.461308 32074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:09.461357 32074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:09.494462 32074 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:34043
I20260927 07:05:09.494503 32201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:34043 every 8 connection(s)
I20260927 07:05:09.495838 32074 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:05:09.499709 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 32074
I20260927 07:05:09.499905 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:05:09.500407 13799 external_mini_cluster.cc:1428] Setting key b86ea41918812bcd5ced74c6e655871f
Sep 27 07:05:09 dist-test-slave-p6hz krb5kdc[31968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492709, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:09.516177 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:09.498284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:57727 (local address 127.13.121.254:32789)
0927 07:05:09.498708 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:09.498712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:09.501929 (+ 3217us) server_negotiation.cc:407] Connection header received
0927 07:05:09.501991 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:09.501996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:09.502078 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:09.502181 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:09.503857 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:09.504620 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:09.506192 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:09.506608 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:09.506724 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:09.510421 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:09.510427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:09.510430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:09.510465 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:09.513002 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:09.513752 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:09.513757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:09.513761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:09.513886 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:09.514329 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:09.514333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:09.514335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:09.514616 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:09.514757 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:09.515571 (+ 814us) server_negotiation.cc:299] Negotiation successful
0927 07:05:09.515820 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":214,"threads_started":1}
I20260927 07:05:09.517215 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:09.498703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:32789 (local address 127.13.121.193:57727)
0927 07:05:09.499469 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:09.499520 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:09.501584 (+ 2064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:09.502452 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:09.502461 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:09.503001 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:09.503664 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:09.503684 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:09.504789 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:09.504793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:09.506043 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:09.506056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:09.506853 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:09.506861 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:09.507098 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:09.507793 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:09.507822 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:09.510169 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:09.513152 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:09.513161 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:09.513180 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:09.513616 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:09.513996 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:09.514001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:09.514005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:09.514201 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:09.514755 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:09.514764 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:09.515129 (+ 365us) client_negotiation.cc:769] Sending connection context
0927 07:05:09.515431 (+ 302us) client_negotiation.cc:240] Negotiation successful
0927 07:05:09.516011 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":251,"threads_started":1}
I20260927 07:05:09.518905 32202 heartbeater.cc:347] Connected to a master server at 127.13.121.254:32789
I20260927 07:05:09.519258 32202 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:09.520053 32202 heartbeater.cc:511] Master 127.13.121.254:32789 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:05:09.522264 32013 ts_manager.cc:194] Registered new tserver with Master: 12e107e2aca2419e81dfd2e7a9624106 (127.13.121.193:34043)
I20260927 07:05:09.523780 32013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:57727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:05:09.554625 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:32789
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:09.879735 32208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:09.880003 32208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:09.880125 32208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:09.887921 32208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:09.888065 32208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:09.888242 32208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:05:09.898483 32208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:32789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:09.899816 32208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:09.901096 32208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:09.913926 32213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:09.913961 32214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:09.913990 32216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:09.914680 32208 server_base.cc:1034] running on GCE node
I20260927 07:05:09.915347 32208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:09.916285 32208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:09.917490 32208 hybrid_clock.cc:648] HybridClock initialized: now 1790492709917418 us; error 52 us; skew 500 ppm
I20260927 07:05:09.917812 32208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:09 dist-test-slave-p6hz krb5kdc[31968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492709, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:09.924865 32208 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:05:09.926541 32208 webserver.cc:460] Webserver started at http://127.13.121.194:34761/ using document root <none> and password file <none>
I20260927 07:05:09.927264 32208 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:09.927376 32208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:09.927675 32208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:09.930545 32208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "9d9a86d153cf4ce1905d5b29e965092a"
format_stamp: "Formatted at 2026-09-27 07:05:09 on dist-test-slave-p6hz"
server_key: "455d7b95252bc9509139ce00a8b63bc0"
server_key_iv: "7f50d72ec07d64bde88425b161a404dd"
server_key_version: "encryptionkey@0"
I20260927 07:05:09.931357 32208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "9d9a86d153cf4ce1905d5b29e965092a"
format_stamp: "Formatted at 2026-09-27 07:05:09 on dist-test-slave-p6hz"
server_key: "455d7b95252bc9509139ce00a8b63bc0"
server_key_iv: "7f50d72ec07d64bde88425b161a404dd"
server_key_version: "encryptionkey@0"
I20260927 07:05:09.936602 32208 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260927 07:05:09.940294 32223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:09.941793 32208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:05:09.941972 32208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "9d9a86d153cf4ce1905d5b29e965092a"
format_stamp: "Formatted at 2026-09-27 07:05:09 on dist-test-slave-p6hz"
server_key: "455d7b95252bc9509139ce00a8b63bc0"
server_key_iv: "7f50d72ec07d64bde88425b161a404dd"
server_key_version: "encryptionkey@0"
I20260927 07:05:09.942179 32208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:09.963632 32208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:09.968163 32208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:09.968478 32208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:09.969470 32208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:09.970762 32208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:09.970834 32208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:09.970924 32208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:09.970965 32208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:10.000979 32208 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:39829
I20260927 07:05:10.001025 32335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:39829 every 8 connection(s)
I20260927 07:05:10.002257 32208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:05:10.007684 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 32208
I20260927 07:05:10.007862 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:05:10.008299 13799 external_mini_cluster.cc:1428] Setting key 6f7751bf0f01e37abb13e42a829c11ea
Sep 27 07:05:10 dist-test-slave-p6hz krb5kdc[31968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492709, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:10.022903 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.004992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:48659 (local address 127.13.121.254:32789)
0927 07:05:10.005235 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.005239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.006426 (+ 1187us) server_negotiation.cc:407] Connection header received
0927 07:05:10.008414 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.008420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.008529 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.008619 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:10.010345 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.011069 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.012671 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.013093 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.013213 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.016802 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:10.016807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:10.016810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:10.016853 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:10.019806 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.020603 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.020609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.020612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.020744 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.021160 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.021164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.021166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.021439 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:10.021582 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.022304 (+ 722us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.022573 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260927 07:05:10.023952 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.005485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:32789 (local address 127.13.121.194:48659)
0927 07:05:10.006244 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.006293 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:10.008125 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:10.008879 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:10.008888 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:10.009517 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:10.010181 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.010204 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.011238 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.011242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.012509 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.012523 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.013325 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.013332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.013569 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.014204 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:10.014233 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:10.016575 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:10.019951 (+ 3376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:10.019961 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:10.019982 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:10.020489 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:10.020846 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:10.020851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:10.020854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:10.021032 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:10.021564 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:10.021572 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:10.021935 (+ 363us) client_negotiation.cc:769] Sending connection context
0927 07:05:10.022189 (+ 254us) client_negotiation.cc:240] Negotiation successful
0927 07:05:10.022741 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":296,"threads_started":1}
I20260927 07:05:10.025527 32336 heartbeater.cc:347] Connected to a master server at 127.13.121.254:32789
I20260927 07:05:10.025846 32336 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:10.026619 32336 heartbeater.cc:511] Master 127.13.121.254:32789 requested a full tablet report, sending...
I20260927 07:05:10.028265 32013 ts_manager.cc:194] Registered new tserver with Master: 9d9a86d153cf4ce1905d5b29e965092a (127.13.121.194:39829)
I20260927 07:05:10.029099 32013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:48659
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:05:10.063817 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:32789
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:10.387961 32341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:10.388259 32341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:10.388360 32341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:10.396117 32341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:10.396270 32341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:10.396392 32341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:05:10.406668 32341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:32789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:10.408023 32341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:10.409349 32341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:10.422870 32346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:10.422910 32349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:10.422928 32347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:10.423651 32341 server_base.cc:1034] running on GCE node
I20260927 07:05:10.424320 32341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:10.425383 32341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:10.426653 32341 hybrid_clock.cc:648] HybridClock initialized: now 1790492710426579 us; error 63 us; skew 500 ppm
I20260927 07:05:10.427004 32341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:10 dist-test-slave-p6hz krb5kdc[31968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492710, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:10.430230 32341 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:05:10.431898 32341 webserver.cc:460] Webserver started at http://127.13.121.195:44123/ using document root <none> and password file <none>
I20260927 07:05:10.432616 32341 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:10.432739 32341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:10.433040 32341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:10.436250 32341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "250ef0bbb24147589bfb8e5051fb57a6"
format_stamp: "Formatted at 2026-09-27 07:05:10 on dist-test-slave-p6hz"
server_key: "08d9aef2653468c0d210435014842274"
server_key_iv: "716b9f1d58d69a5b894b18c347e0ad19"
server_key_version: "encryptionkey@0"
I20260927 07:05:10.437088 32341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "250ef0bbb24147589bfb8e5051fb57a6"
format_stamp: "Formatted at 2026-09-27 07:05:10 on dist-test-slave-p6hz"
server_key: "08d9aef2653468c0d210435014842274"
server_key_iv: "716b9f1d58d69a5b894b18c347e0ad19"
server_key_version: "encryptionkey@0"
I20260927 07:05:10.442802 32341 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:05:10.446761 32356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:10.448318 32341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:10.448501 32341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "250ef0bbb24147589bfb8e5051fb57a6"
format_stamp: "Formatted at 2026-09-27 07:05:10 on dist-test-slave-p6hz"
server_key: "08d9aef2653468c0d210435014842274"
server_key_iv: "716b9f1d58d69a5b894b18c347e0ad19"
server_key_version: "encryptionkey@0"
I20260927 07:05:10.448725 32341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:10.482841 32341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:10.487665 32341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:10.487949 32341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:10.489003 32341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:10.490413 32341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:10.490494 32341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:10.490623 32341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:10.490676 32341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:10.521514 32341 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:36565
I20260927 07:05:10.521544 32468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:36565 every 8 connection(s)
I20260927 07:05:10.522854 32341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:05:10.527364 32202 heartbeater.cc:503] Master 127.13.121.254:32789 was elected leader, sending a full tablet report...
I20260927 07:05:10.528090 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 32341
I20260927 07:05:10.528326 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:05:10.528901 13799 external_mini_cluster.cc:1428] Setting key 22f384d84f1e42eaf83a697a3eae085e
Sep 27 07:05:10 dist-test-slave-p6hz krb5kdc[31968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492710, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:10.546335 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.525552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:49833 (local address 127.13.121.254:32789)
0927 07:05:10.526071 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.526078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.529610 (+ 3532us) server_negotiation.cc:407] Connection header received
0927 07:05:10.530618 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.530623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.530712 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.530821 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:10.532517 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.533350 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.534964 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.535436 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.535551 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.539287 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:10.539293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:10.539296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:10.539334 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:10.542254 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.543418 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.543480 (+ 62us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.543483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.543677 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.544335 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.544339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.544341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.544675 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:10.544802 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.545667 (+ 865us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.545889 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":235,"threads_started":1}
I20260927 07:05:10.547490 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.526135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:32789 (local address 127.13.121.195:49833)
0927 07:05:10.527932 (+ 1797us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.527999 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:10.530253 (+ 2254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:10.531053 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:10.531098 (+ 45us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:10.531744 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:10.532340 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.532359 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.533506 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.533510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.534807 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.534821 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.535653 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.535659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.535867 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.536565 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:10.536597 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:10.538985 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:10.542474 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:10.542484 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:10.542502 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:10.543281 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:10.543820 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:10.543825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:10.543860 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:10.544217 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:10.544814 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:10.544821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:10.545262 (+ 441us) client_negotiation.cc:769] Sending connection context
0927 07:05:10.545529 (+ 267us) client_negotiation.cc:240] Negotiation successful
0927 07:05:10.546072 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1336,"spinlock_wait_cycles":20224,"thread_start_us":1260,"threads_started":1}
I20260927 07:05:10.548911 32469 heartbeater.cc:347] Connected to a master server at 127.13.121.254:32789
I20260927 07:05:10.549243 32469 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:10.549962 32469 heartbeater.cc:511] Master 127.13.121.254:32789 requested a full tablet report, sending...
I20260927 07:05:10.551636 32013 ts_manager.cc:194] Registered new tserver with Master: 250ef0bbb24147589bfb8e5051fb57a6 (127.13.121.195:36565)
I20260927 07:05:10.552672 32013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:49833
I20260927 07:05:10.557019 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:05:10.574560 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.561272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.13.121.254:32789)
0927 07:05:10.561553 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.561560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.561843 (+ 283us) server_negotiation.cc:407] Connection header received
0927 07:05:10.562006 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.562011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.562098 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.562195 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:10.563677 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.564416 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.565759 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.566193 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.566296 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.569175 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:10.569180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:10.569184 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:10.569224 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:10.571934 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.572772 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.572776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.572779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.572916 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.573310 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.573314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.573316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.573648 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:10.573850 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.574119 (+ 269us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.574268 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260927 07:05:10.578649 32013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:05:10.585457 32013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:05:10.624222 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.597768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:36565 (local address 127.0.0.1:48504)
0927 07:05:10.598677 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.598721 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:10.598879 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:10.615424 (+ 16545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:10.615432 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:10.615470 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:10.616140 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.616155 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.618381 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.618386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.620988 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.621003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.622894 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.622900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.623218 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.623288 (+ 70us) client_negotiation.cc:769] Sending connection context
0927 07:05:10.623437 (+ 149us) client_negotiation.cc:240] Negotiation successful
0927 07:05:10.623653 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":79,"spinlock_wait_cycles":1920,"thread_start_us":550,"threads_started":1}
I20260927 07:05:10.624275 32485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.598590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.13.121.193:34043)
0927 07:05:10.610446 (+ 11856us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.610457 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.613141 (+ 2684us) server_negotiation.cc:407] Connection header received
0927 07:05:10.613288 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.613300 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.613659 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.613927 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:10.616426 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.618140 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.621056 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.622632 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.622852 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.623087 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.623736 (+ 649us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.623937 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11653,"thread_start_us":11486,"threads_started":1}
I20260927 07:05:10.624859 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.598172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:34043 (local address 127.0.0.1:56848)
0927 07:05:10.612716 (+ 14544us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.612743 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:10.612967 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:10.615462 (+ 2495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:10.615468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:10.615497 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:10.616129 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.616145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.618321 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.618326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.620871 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.620884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.623322 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.623329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.623501 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.623533 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:05:10.623615 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 07:05:10.623736 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14351,"mutex_wait_us":105,"thread_start_us":19383,"threads_started":1}
I20260927 07:05:10.625120 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.598102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.13.121.195:36565)
0927 07:05:10.614660 (+ 16558us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.614670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.614711 (+ 41us) server_negotiation.cc:407] Connection header received
0927 07:05:10.614822 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.614830 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.615096 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.616379 (+ 1283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:10.616722 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.618233 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.621171 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.622631 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.624247 (+ 1616us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.624449 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.624519 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.624681 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16357,"thread_start_us":15948,"threads_started":1}
I20260927 07:05:10.626405 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.599082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:39829 (local address 127.0.0.1:42270)
0927 07:05:10.618157 (+ 19075us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.618178 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:10.618317 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:10.619410 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:10.619415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:10.619443 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:10.619910 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.619923 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.621688 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.621694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.623692 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.623707 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.625591 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.625597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.625776 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.625809 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 07:05:10.625898 (+ 89us) client_negotiation.cc:240] Negotiation successful
0927 07:05:10.626032 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18928,"mutex_wait_us":1,"thread_start_us":15117,"threads_started":1}
I20260927 07:05:10.629220 32404 tablet_service.cc:1467] Processing CreateTablet for tablet 3e4b40e81c5046219cbfb04e8259a2c6 (DEFAULT_TABLE table=test-table [id=96c3dda299d0459a89e70595e56d47ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:10.629511 32137 tablet_service.cc:1467] Processing CreateTablet for tablet 3e4b40e81c5046219cbfb04e8259a2c6 (DEFAULT_TABLE table=test-table [id=96c3dda299d0459a89e70595e56d47ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:10.630543 32404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e4b40e81c5046219cbfb04e8259a2c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:10.630635 32137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e4b40e81c5046219cbfb04e8259a2c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:10.631538 32487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.599402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.13.121.194:39829)
0927 07:05:10.616245 (+ 16843us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.616253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.618447 (+ 2194us) server_negotiation.cc:407] Connection header received
0927 07:05:10.618563 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.618571 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.618981 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.619431 (+ 450us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:10.620118 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.621549 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.623889 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.625383 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.625546 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.625709 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.625969 (+ 260us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.626133 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16636,"thread_start_us":16841,"threads_started":1}
I20260927 07:05:10.635473 32271 tablet_service.cc:1467] Processing CreateTablet for tablet 3e4b40e81c5046219cbfb04e8259a2c6 (DEFAULT_TABLE table=test-table [id=96c3dda299d0459a89e70595e56d47ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:10.637079 32271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e4b40e81c5046219cbfb04e8259a2c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:10.642658 32488 tablet_bootstrap.cc:492] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: Bootstrap starting.
I20260927 07:05:10.644907 32489 tablet_bootstrap.cc:492] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: Bootstrap starting.
I20260927 07:05:10.646929 32488 tablet_bootstrap.cc:654] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:10.648300 32488 log.cc:824] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:10.648561 32489 tablet_bootstrap.cc:654] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:10.649618 32489 log.cc:824] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:10.651458 32488 tablet_bootstrap.cc:492] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: No bootstrap required, opened a new log
I20260927 07:05:10.651574 32492 tablet_bootstrap.cc:492] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: Bootstrap starting.
I20260927 07:05:10.652187 32488 ts_tablet_manager.cc:1397] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260927 07:05:10.652627 32489 tablet_bootstrap.cc:492] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: No bootstrap required, opened a new log
I20260927 07:05:10.652932 32489 ts_tablet_manager.cc:1397] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260927 07:05:10.654649 32492 tablet_bootstrap.cc:654] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:10.656349 32492 log.cc:824] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:10.658957 32492 tablet_bootstrap.cc:492] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: No bootstrap required, opened a new log
I20260927 07:05:10.659243 32492 ts_tablet_manager.cc:1397] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260927 07:05:10.662617 32488 raft_consensus.cc:348] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } }
I20260927 07:05:10.663100 32488 raft_consensus.cc:374] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:10.663264 32488 raft_consensus.cc:729] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e107e2aca2419e81dfd2e7a9624106, State: Initialized, Role: FOLLOWER
I20260927 07:05:10.663166 32489 raft_consensus.cc:348] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } }
I20260927 07:05:10.663568 32489 raft_consensus.cc:374] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:10.663663 32489 raft_consensus.cc:729] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 250ef0bbb24147589bfb8e5051fb57a6, State: Initialized, Role: FOLLOWER
I20260927 07:05:10.664000 32488 consensus_queue.cc:260] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } }
I20260927 07:05:10.664173 32489 consensus_queue.cc:260] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } }
I20260927 07:05:10.665843 32488 ts_tablet_manager.cc:1428] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260927 07:05:10.666750 32469 heartbeater.cc:503] Master 127.13.121.254:32789 was elected leader, sending a full tablet report...
I20260927 07:05:10.667361 32489 ts_tablet_manager.cc:1428] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260927 07:05:10.669155 32492 raft_consensus.cc:348] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } }
I20260927 07:05:10.669550 32492 raft_consensus.cc:374] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:10.669656 32492 raft_consensus.cc:729] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d9a86d153cf4ce1905d5b29e965092a, State: Initialized, Role: FOLLOWER
I20260927 07:05:10.670238 32492 consensus_queue.cc:260] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } }
I20260927 07:05:10.671242 32336 heartbeater.cc:503] Master 127.13.121.254:32789 was elected leader, sending a full tablet report...
I20260927 07:05:10.671993 32492 ts_tablet_manager.cc:1428] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: Time spent starting tablet: real 0.013s user 0.010s sys 0.000s
I20260927 07:05:10.687666 32496 raft_consensus.cc:482] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:05:10.687953 32496 raft_consensus.cc:504] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } }
I20260927 07:05:10.689558 32496 leader_election.cc:288] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 250ef0bbb24147589bfb8e5051fb57a6 (127.13.121.195:36565), 12e107e2aca2419e81dfd2e7a9624106 (127.13.121.193:34043)
I20260927 07:05:10.696241 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.690039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:49853 (local address 127.13.121.195:36565)
0927 07:05:10.690243 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.690250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.690958 (+ 708us) server_negotiation.cc:407] Connection header received
0927 07:05:10.691110 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.691114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.691210 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.691287 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:10.691908 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.692603 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.694487 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.695290 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.695413 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.695507 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.695866 (+ 359us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.695988 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:05:10.696326 32498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.689971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:36565 (local address 127.13.121.194:49853)
0927 07:05:10.690842 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.690858 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:10.690989 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:10.691319 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:10.691323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:10.691379 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:10.691760 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.691769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.692739 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.692742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.694336 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.694350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.695503 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.695510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.695673 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.695727 (+ 54us) client_negotiation.cc:769] Sending connection context
0927 07:05:10.695809 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 07:05:10.695928 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":48,"spinlock_wait_cycles":40832,"thread_start_us":489,"threads_started":1}
I20260927 07:05:10.696419 32485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.690154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:52861 (local address 127.13.121.193:34043)
0927 07:05:10.690405 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.690411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.690908 (+ 497us) server_negotiation.cc:407] Connection header received
0927 07:05:10.691073 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.691078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.691206 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.691291 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:10.691886 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.692595 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.694511 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.695325 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.695476 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.695546 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.696008 (+ 462us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.696190 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260927 07:05:10.697177 32497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.689857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:34043 (local address 127.13.121.194:52861)
0927 07:05:10.690790 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.690806 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:10.690935 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:10.691316 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:10.691321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:10.691348 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:10.691745 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.691756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.692716 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.692721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.694358 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:10.694369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.695540 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.695546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:10.695807 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.695844 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 07:05:10.696645 (+ 801us) client_negotiation.cc:240] Negotiation successful
0927 07:05:10.696764 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":79,"thread_start_us":547,"threads_started":1}
I20260927 07:05:10.697546 32424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4b40e81c5046219cbfb04e8259a2c6" candidate_uuid: "9d9a86d153cf4ce1905d5b29e965092a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "250ef0bbb24147589bfb8e5051fb57a6" is_pre_election: true
I20260927 07:05:10.698108 32424 raft_consensus.cc:2454] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d9a86d153cf4ce1905d5b29e965092a in term 0.
I20260927 07:05:10.698210 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4b40e81c5046219cbfb04e8259a2c6" candidate_uuid: "9d9a86d153cf4ce1905d5b29e965092a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e107e2aca2419e81dfd2e7a9624106" is_pre_election: true
I20260927 07:05:10.698851 32157 raft_consensus.cc:2454] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d9a86d153cf4ce1905d5b29e965092a in term 0.
I20260927 07:05:10.698894 32227 leader_election.cc:302] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 250ef0bbb24147589bfb8e5051fb57a6, 9d9a86d153cf4ce1905d5b29e965092a; no voters:
I20260927 07:05:10.699342 32496 raft_consensus.cc:2781] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:05:10.699506 32496 raft_consensus.cc:482] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:05:10.699616 32496 raft_consensus.cc:3037] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:10.701968 32496 raft_consensus.cc:504] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } }
I20260927 07:05:10.702589 32496 leader_election.cc:288] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [CANDIDATE]: Term 1 election: Requested vote from peers 250ef0bbb24147589bfb8e5051fb57a6 (127.13.121.195:36565), 12e107e2aca2419e81dfd2e7a9624106 (127.13.121.193:34043)
I20260927 07:05:10.703114 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4b40e81c5046219cbfb04e8259a2c6" candidate_uuid: "9d9a86d153cf4ce1905d5b29e965092a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e107e2aca2419e81dfd2e7a9624106"
I20260927 07:05:10.703254 32424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4b40e81c5046219cbfb04e8259a2c6" candidate_uuid: "9d9a86d153cf4ce1905d5b29e965092a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "250ef0bbb24147589bfb8e5051fb57a6"
I20260927 07:05:10.703373 32157 raft_consensus.cc:3037] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:10.703455 32424 raft_consensus.cc:3037] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:10.705535 32157 raft_consensus.cc:2454] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d9a86d153cf4ce1905d5b29e965092a in term 1.
I20260927 07:05:10.705569 32424 raft_consensus.cc:2454] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d9a86d153cf4ce1905d5b29e965092a in term 1.
I20260927 07:05:10.705920 32224 leader_election.cc:302] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e107e2aca2419e81dfd2e7a9624106, 9d9a86d153cf4ce1905d5b29e965092a; no voters:
I20260927 07:05:10.706209 32496 raft_consensus.cc:2781] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:10.706598 32496 raft_consensus.cc:686] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 1 LEADER]: Becoming Leader. State: Replica: 9d9a86d153cf4ce1905d5b29e965092a, State: Running, Role: LEADER
I20260927 07:05:10.707050 32496 consensus_queue.cc:237] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } }
I20260927 07:05:10.711813 32013 catalog_manager.cc:5495] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d9a86d153cf4ce1905d5b29e965092a (127.13.121.194). New cstate: current_term: 1 leader_uuid: "9d9a86d153cf4ce1905d5b29e965092a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d9a86d153cf4ce1905d5b29e965092a" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39829 } health_report { overall_health: HEALTHY } } }
I20260927 07:05:10.728621 32487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.722723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.13.121.194:39829)
0927 07:05:10.722936 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.722943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.722964 (+ 21us) server_negotiation.cc:407] Connection header received
0927 07:05:10.723204 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.723208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.723292 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.723414 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:10.723806 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.724568 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.725952 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.726352 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.726473 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.726978 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:10.727653 (+ 675us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:10.727791 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.728267 (+ 476us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.728317 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:05:10.736902 32157 raft_consensus.cc:1261] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [term 1 FOLLOWER]: Refusing update from remote peer 9d9a86d153cf4ce1905d5b29e965092a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:10.736902 32424 raft_consensus.cc:1261] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [term 1 FOLLOWER]: Refusing update from remote peer 9d9a86d153cf4ce1905d5b29e965092a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:10.737927 32499 consensus_queue.cc:1035] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [LEADER]: Connected to new peer: Peer: permanent_uuid: "12e107e2aca2419e81dfd2e7a9624106" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 34043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:10.738287 32496 consensus_queue.cc:1035] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [LEADER]: Connected to new peer: Peer: permanent_uuid: "250ef0bbb24147589bfb8e5051fb57a6" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:10.746991 32502 mvcc.cc:205] Tried to move back new op lower bound from 7333858143167762432 to 7333858143060074496. Current Snapshot: MvccSnapshot[applied={T|T < 7333858143167762432}]
W20260927 07:05:10.748088 32203 tablet.cc:2369] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:05:10.748936 32503 mvcc.cc:205] Tried to move back new op lower bound from 7333858143167762432 to 7333858143060074496. Current Snapshot: MvccSnapshot[applied={T|T < 7333858143167762432}]
I20260927 07:05:10.766480 32013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:10.766793 32013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:10.771178 32013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32eb0c4e762e4341831a3b5f691b5b19: Sending DeleteTablet for 3 replicas of tablet 3e4b40e81c5046219cbfb04e8259a2c6
I20260927 07:05:10.773069 32404 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e4b40e81c5046219cbfb04e8259a2c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:10 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:48504
I20260927 07:05:10.773154 32137 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e4b40e81c5046219cbfb04e8259a2c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:10 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:56848
I20260927 07:05:10.773289 32271 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e4b40e81c5046219cbfb04e8259a2c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:10 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:42270
I20260927 07:05:10.778430 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 32074
I20260927 07:05:10.785548 32513 tablet_replica.cc:330] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: stopping tablet replica
I20260927 07:05:10.786263 32514 tablet_replica.cc:330] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: stopping tablet replica
I20260927 07:05:10.786996 32514 raft_consensus.cc:2229] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:05:10.787544 32513 raft_consensus.cc:2229] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:10.788197 32513 raft_consensus.cc:2258] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:10.789127 32514 raft_consensus.cc:2258] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:10.790501 32512 tablet_replica.cc:330] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: stopping tablet replica
I20260927 07:05:10.792016 32513 ts_tablet_manager.cc:1905] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:10.792032 32514 ts_tablet_manager.cc:1905] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:10.798717 32512 raft_consensus.cc:2229] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:10.799458 32513 ts_tablet_manager.cc:1918] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:10.799724 32513 log.cc:1196] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/3e4b40e81c5046219cbfb04e8259a2c6
I20260927 07:05:10.800076 32514 ts_tablet_manager.cc:1918] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:10.800334 32513 ts_tablet_manager.cc:1939] T 3e4b40e81c5046219cbfb04e8259a2c6 P 250ef0bbb24147589bfb8e5051fb57a6: Deleting consensus metadata
I20260927 07:05:10.800392 32514 log.cc:1196] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/3e4b40e81c5046219cbfb04e8259a2c6
I20260927 07:05:10.800980 32514 ts_tablet_manager.cc:1939] T 3e4b40e81c5046219cbfb04e8259a2c6 P 9d9a86d153cf4ce1905d5b29e965092a: Deleting consensus metadata
Sep 27 07:05:10 dist-test-slave-p6hz krb5kdc[31968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:05:10.802498 32000 catalog_manager.cc:4841] TS 250ef0bbb24147589bfb8e5051fb57a6 (127.13.121.195:36565): tablet 3e4b40e81c5046219cbfb04e8259a2c6 (table test-table [id=96c3dda299d0459a89e70595e56d47ad]) successfully deleted
I20260927 07:05:10.802693 32512 raft_consensus.cc:2258] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:10.802862 32000 catalog_manager.cc:4841] TS 9d9a86d153cf4ce1905d5b29e965092a (127.13.121.194:39829): tablet 3e4b40e81c5046219cbfb04e8259a2c6 (table test-table [id=96c3dda299d0459a89e70595e56d47ad]) successfully deleted
I20260927 07:05:10.804955 32512 ts_tablet_manager.cc:1905] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:10.811335 32485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.779297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56860 (local address 127.13.121.193:34043)
0927 07:05:10.779486 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.779493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.791320 (+ 11827us) server_negotiation.cc:407] Connection header received
0927 07:05:10.791645 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.791651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.791740 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.791815 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:10.793658 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.794768 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.796586 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.797282 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.797385 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.803833 (+ 6448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:10.803847 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:10.803864 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:10.803918 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:10.807943 (+ 4025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.808901 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.808910 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.808913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.809069 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.809502 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.809507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.809511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.809999 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:10.810345 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.810659 (+ 314us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.810910 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:05:10.811997 32197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:10.902676 32512 ts_tablet_manager.cc:1918] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:10.903028 32512 log.cc:1196] T 3e4b40e81c5046219cbfb04e8259a2c6 P 12e107e2aca2419e81dfd2e7a9624106: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/3e4b40e81c5046219cbfb04e8259a2c6
I20260927 07:05:10.903412 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 32074
W20260927 07:05:10.918169 31997 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.121.193:34043 (error 108)
I20260927 07:05:10.918421 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 32208
I20260927 07:05:10.920276 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.919743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:34043 (local address 127.0.0.1:56872)
0927 07:05:10.919944 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.920071 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:34043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260927 07:05:10.920568 31997 catalog_manager.cc:4584] TS 12e107e2aca2419e81dfd2e7a9624106 (127.13.121.193:34043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e4b40e81c5046219cbfb04e8259a2c6: Network error: Client connection negotiation failed: client connection to 127.13.121.193:34043: connect: Connection refused (error 111)
Sep 27 07:05:10 dist-test-slave-p6hz krb5kdc[31968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:05:10.935832 32487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.919272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.13.121.194:39829)
0927 07:05:10.919463 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:10.919468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:10.919545 (+ 77us) server_negotiation.cc:407] Connection header received
0927 07:05:10.919732 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:10.919739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:10.919821 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:10.919929 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:10.921338 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.922144 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.923482 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:10.923936 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:10.924049 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:10.928708 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:10.928726 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:10.928743 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:10.928791 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:10.932394 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.933445 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.933453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.933459 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.933633 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:10.934138 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:10.934142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:10.934144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:10.934619 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:10.934894 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:10.935315 (+ 421us) server_negotiation.cc:299] Negotiation successful
0927 07:05:10.935503 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:05:10.936590 32331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:10.974663 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.974167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:34043 (local address 127.0.0.1:56886)
0927 07:05:10.974391 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.974493 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:34043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260927 07:05:10.975728 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:10.975354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:34043 (local address 127.0.0.1:56890)
0927 07:05:10.975504 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:10.975603 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:34043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 07:05:11.036172 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:11.035641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:34043 (local address 127.0.0.1:56896)
0927 07:05:11.035880 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:11.035971 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:34043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260927 07:05:11.037317 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:11.036844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:34043 (local address 127.0.0.1:56910)
0927 07:05:11.037039 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:11.037155 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:34043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 07:05:11.050844 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 32208
I20260927 07:05:11.066578 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 32341
Sep 27 07:05:11 dist-test-slave-p6hz krb5kdc[31968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:05:11.082749 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:11.067419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48506 (local address 127.13.121.195:36565)
0927 07:05:11.067612 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:11.067617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:11.067637 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:05:11.067824 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:11.067830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:11.067917 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:11.068010 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:11.069298 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:11.070075 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:11.071594 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:11.072022 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:11.072138 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:11.076802 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:11.076820 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:11.076836 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:11.076879 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:11.079803 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:11.080570 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:11.080579 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:11.080581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:11.080751 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:11.081187 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:11.081192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:11.081194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:11.081623 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:11.081864 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:11.082243 (+ 379us) server_negotiation.cc:299] Negotiation successful
0927 07:05:11.082474 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 07:05:11.083478 32464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:11.117869 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:11.117338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:34043 (local address 127.0.0.1:56918)
0927 07:05:11.117552 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:11.117666 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:34043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260927 07:05:11.118922 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:11.118502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:34043 (local address 127.0.0.1:56932)
0927 07:05:11.118720 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:11.118807 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:34043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260927 07:05:11.173421 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 32341
I20260927 07:05:11.187971 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 31981
I20260927 07:05:11.189142 32043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:11.301402 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 31981
2026-09-27T07:05:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4120 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-27T07:05:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:11Z Disabled control of system clock
I20260927 07:05:11.374634 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:33551
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:32889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:33551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:05:11.710196 32524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:11.710544 32524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:11.710660 32524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:11.718689 32524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:11.718854 32524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:11.718920 32524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:11.719002 32524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:11.719071 32524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:11.728856 32524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:32889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:33551
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:33551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:11.730171 32524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:11.732052 32524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:11.743618 32530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:11.743655 32529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:11.743637 32532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:11.744306 32524 server_base.cc:1034] running on GCE node
I20260927 07:05:11.745110 32524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:11.746588 32524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:11.747821 32524 hybrid_clock.cc:648] HybridClock initialized: now 1790492711747761 us; error 50 us; skew 500 ppm
I20260927 07:05:11.748270 32524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:11.753247 32524 webserver.cc:460] Webserver started at http://127.13.121.254:33127/ using document root <none> and password file <none>
I20260927 07:05:11.754231 32524 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:11.754367 32524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:11.754873 32524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:11.759537 32524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "d329bbabff5a4725a11152b5d1cc600f"
format_stamp: "Formatted at 2026-09-27 07:05:11 on dist-test-slave-p6hz"
server_key: "69e22adb34f8987bdd630a03fad54768"
server_key_iv: "e818ea017d79367b30b83fa8202bf5c2"
server_key_version: "encryptionkey@0"
I20260927 07:05:11.760567 32524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "d329bbabff5a4725a11152b5d1cc600f"
format_stamp: "Formatted at 2026-09-27 07:05:11 on dist-test-slave-p6hz"
server_key: "69e22adb34f8987bdd630a03fad54768"
server_key_iv: "e818ea017d79367b30b83fa8202bf5c2"
server_key_version: "encryptionkey@0"
I20260927 07:05:11.768492 32524 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260927 07:05:11.773044 32538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:11.774852 32524 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260927 07:05:11.775058 32524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "d329bbabff5a4725a11152b5d1cc600f"
format_stamp: "Formatted at 2026-09-27 07:05:11 on dist-test-slave-p6hz"
server_key: "69e22adb34f8987bdd630a03fad54768"
server_key_iv: "e818ea017d79367b30b83fa8202bf5c2"
server_key_version: "encryptionkey@0"
I20260927 07:05:11.775358 32524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:11.800140 32524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:11.801203 32524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:11.801514 32524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:11.819129 32524 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:33551
I20260927 07:05:11.819238 32589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:33551 every 8 connection(s)
I20260927 07:05:11.820621 32524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:11.824908 32590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:11.828243 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 32524
I20260927 07:05:11.828433 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:11.828972 13799 external_mini_cluster.cc:1428] Setting key 43c800f11ed2b251f7492029d0ff6d42
I20260927 07:05:11.837774 32590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f: Bootstrap starting.
I20260927 07:05:11.841251 32590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:11.842522 32590 log.cc:824] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:11.843389 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:11.831323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.13.121.254:33551)
0927 07:05:11.832395 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:11.832407 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:11.832452 (+ 45us) server_negotiation.cc:407] Connection header received
0927 07:05:11.834195 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:11.834224 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:11.834766 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:11.835200 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:11.835823 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:11.837101 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:11.838525 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:11.839312 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:11.839499 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:11.840616 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:11.840625 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:11.840640 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:11.841225 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:11.841326 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:11.841360 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:11.841470 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:11.841807 (+ 337us) server_negotiation.cc:299] Negotiation successful
0927 07:05:11.842161 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":376,"threads_started":1}
I20260927 07:05:11.846064 32590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f: No bootstrap required, opened a new log
I20260927 07:05:11.853739 32590 raft_consensus.cc:348] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d329bbabff5a4725a11152b5d1cc600f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33551 } }
I20260927 07:05:11.854146 32590 raft_consensus.cc:374] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:11.854223 32590 raft_consensus.cc:729] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d329bbabff5a4725a11152b5d1cc600f, State: Initialized, Role: FOLLOWER
I20260927 07:05:11.854765 32590 consensus_queue.cc:260] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d329bbabff5a4725a11152b5d1cc600f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33551 } }
I20260927 07:05:11.854975 32590 raft_consensus.cc:388] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:11.855074 32590 raft_consensus.cc:482] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:11.855248 32590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:11.857442 32590 raft_consensus.cc:504] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d329bbabff5a4725a11152b5d1cc600f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33551 } }
I20260927 07:05:11.857950 32590 leader_election.cc:302] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d329bbabff5a4725a11152b5d1cc600f; no voters:
I20260927 07:05:11.858584 32590 leader_election.cc:288] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:11.858788 32595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:11.859614 32595 raft_consensus.cc:686] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [term 1 LEADER]: Becoming Leader. State: Replica: d329bbabff5a4725a11152b5d1cc600f, State: Running, Role: LEADER
I20260927 07:05:11.860277 32595 consensus_queue.cc:237] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d329bbabff5a4725a11152b5d1cc600f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33551 } }
I20260927 07:05:11.860997 32590 sys_catalog.cc:564] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:11.863343 32597 sys_catalog.cc:455] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d329bbabff5a4725a11152b5d1cc600f. Latest consensus state: current_term: 1 leader_uuid: "d329bbabff5a4725a11152b5d1cc600f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d329bbabff5a4725a11152b5d1cc600f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33551 } } }
I20260927 07:05:11.863615 32596 sys_catalog.cc:455] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d329bbabff5a4725a11152b5d1cc600f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d329bbabff5a4725a11152b5d1cc600f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33551 } } }
I20260927 07:05:11.863823 32597 sys_catalog.cc:458] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:11.863945 32596 sys_catalog.cc:458] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:11.865456 32603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:11.870057 32603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:11.879882 32603 catalog_manager.cc:1347] Generated new cluster ID: 19d2c8ba89bd434d9f175e5a5ec19eec
I20260927 07:05:11.880024 32603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:11.892323 32603 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:11.894363 32603 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:11.907214 32603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f: Generated new TSK 0
I20260927 07:05:11.908476 32603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:11.919351 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33551
--builtin_ntp_servers=127.13.121.212:32889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:05:12.261169 32614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:12.261482 32614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:12.261613 32614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:12.269204 32614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:12.269315 32614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:12.269459 32614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:05:12.278726 32614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:32889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:12.280011 32614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:12.281867 32614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:12.294812 32619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:12.294879 32622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:12.294783 32620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:12.295758 32614 server_base.cc:1034] running on GCE node
I20260927 07:05:12.296353 32614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:12.297356 32614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:12.298601 32614 hybrid_clock.cc:648] HybridClock initialized: now 1790492712298535 us; error 51 us; skew 500 ppm
I20260927 07:05:12.298941 32614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:12.301379 32614 webserver.cc:460] Webserver started at http://127.13.121.193:44313/ using document root <none> and password file <none>
I20260927 07:05:12.302091 32614 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:12.302199 32614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:12.302510 32614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:12.305583 32614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "5c0d18af08dc4216b8ab57882568955d"
format_stamp: "Formatted at 2026-09-27 07:05:12 on dist-test-slave-p6hz"
server_key: "a5964012bebafcc3d44ebdabe2c5c0cf"
server_key_iv: "6cf974efae19008c2b49efad68207fb7"
server_key_version: "encryptionkey@0"
I20260927 07:05:12.306409 32614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "5c0d18af08dc4216b8ab57882568955d"
format_stamp: "Formatted at 2026-09-27 07:05:12 on dist-test-slave-p6hz"
server_key: "a5964012bebafcc3d44ebdabe2c5c0cf"
server_key_iv: "6cf974efae19008c2b49efad68207fb7"
server_key_version: "encryptionkey@0"
I20260927 07:05:12.311875 32614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260927 07:05:12.315683 32628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:12.317281 32614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260927 07:05:12.317467 32614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "5c0d18af08dc4216b8ab57882568955d"
format_stamp: "Formatted at 2026-09-27 07:05:12 on dist-test-slave-p6hz"
server_key: "a5964012bebafcc3d44ebdabe2c5c0cf"
server_key_iv: "6cf974efae19008c2b49efad68207fb7"
server_key_version: "encryptionkey@0"
I20260927 07:05:12.317692 32614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:12.342469 32614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:12.343436 32614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:12.343744 32614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:12.344794 32614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:12.346342 32614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:12.346446 32614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:12.346560 32614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:12.346632 32614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:12.377350 32614 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:41627
I20260927 07:05:12.377406 32740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:41627 every 8 connection(s)
I20260927 07:05:12.378631 32614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:05:12.383606 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 32614
I20260927 07:05:12.383759 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:05:12.384176 13799 external_mini_cluster.cc:1428] Setting key 8fbc6a389490d6e9fe649781c8efeae5
I20260927 07:05:12.387041 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33551
--builtin_ntp_servers=127.13.121.212:32889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:05:12.392004 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:12.381320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:46241 (local address 127.13.121.254:33551)
0927 07:05:12.381758 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:12.381763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:12.383074 (+ 1311us) server_negotiation.cc:407] Connection header received
0927 07:05:12.385065 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:12.385071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:12.385162 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:12.385256 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:12.386539 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:12.387377 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:12.389013 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:12.389430 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:12.389542 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:12.390979 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:12.390984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:12.390988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:12.391010 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:12.391062 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:12.391076 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:12.391194 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:12.391425 (+ 231us) server_negotiation.cc:299] Negotiation successful
0927 07:05:12.391637 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":227,"threads_started":1}
I20260927 07:05:12.393209 32744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:12.381915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:33551 (local address 127.13.121.193:46241)
0927 07:05:12.382853 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:12.382896 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:12.384740 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:12.385518 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:12.385527 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:12.385648 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:12.386360 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:12.386381 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:12.387545 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:12.387549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:12.388855 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:12.388869 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:12.389677 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:12.389686 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:12.389919 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:12.390626 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:05:12.390629 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:12.390675 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:05:12.390772 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:12.391199 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:12.391213 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:12.391222 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 07:05:12.391402 (+ 180us) client_negotiation.cc:240] Negotiation successful
0927 07:05:12.392021 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":272,"threads_started":1}
I20260927 07:05:12.394944 32741 heartbeater.cc:347] Connected to a master server at 127.13.121.254:33551
I20260927 07:05:12.395308 32741 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:12.396095 32741 heartbeater.cc:511] Master 127.13.121.254:33551 requested a full tablet report, sending...
I20260927 07:05:12.398190 32555 ts_manager.cc:194] Registered new tserver with Master: 5c0d18af08dc4216b8ab57882568955d (127.13.121.193:41627)
I20260927 07:05:12.399804 32555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.193:46241
W20260927 07:05:12.711874 32745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:12.712191 32745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:12.712322 32745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:12.719715 32745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:12.719834 32745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:12.719980 32745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:05:12.729274 32745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:32889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:12.730572 32745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:12.732339 32745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:12.745059 32750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:12.745071 32751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:12.745078 32753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:12.745890 32745 server_base.cc:1034] running on GCE node
I20260927 07:05:12.746479 32745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:12.747547 32745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:12.748764 32745 hybrid_clock.cc:648] HybridClock initialized: now 1790492712748713 us; error 40 us; skew 500 ppm
I20260927 07:05:12.749100 32745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:12.751463 32745 webserver.cc:460] Webserver started at http://127.13.121.194:39983/ using document root <none> and password file <none>
I20260927 07:05:12.752143 32745 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:12.752234 32745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:12.752535 32745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:12.755487 32745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "7c15ebc5c63e4e9290c11651c20a4584"
format_stamp: "Formatted at 2026-09-27 07:05:12 on dist-test-slave-p6hz"
server_key: "fdccf9dc51525dc34393ce59e674bc61"
server_key_iv: "97f0c797b35f9dfae512414b981fdbf2"
server_key_version: "encryptionkey@0"
I20260927 07:05:12.756276 32745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "7c15ebc5c63e4e9290c11651c20a4584"
format_stamp: "Formatted at 2026-09-27 07:05:12 on dist-test-slave-p6hz"
server_key: "fdccf9dc51525dc34393ce59e674bc61"
server_key_iv: "97f0c797b35f9dfae512414b981fdbf2"
server_key_version: "encryptionkey@0"
I20260927 07:05:12.761808 32745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:05:12.765547 32759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:12.766993 32745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260927 07:05:12.767252 32745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "7c15ebc5c63e4e9290c11651c20a4584"
format_stamp: "Formatted at 2026-09-27 07:05:12 on dist-test-slave-p6hz"
server_key: "fdccf9dc51525dc34393ce59e674bc61"
server_key_iv: "97f0c797b35f9dfae512414b981fdbf2"
server_key_version: "encryptionkey@0"
I20260927 07:05:12.767513 32745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:12.796859 32745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:12.797732 32745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:12.798014 32745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:12.798980 32745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:12.800469 32745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:12.800563 32745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:12.800673 32745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:12.800740 32745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:12.831456 32745 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:42199
I20260927 07:05:12.831494 403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:42199 every 8 connection(s)
I20260927 07:05:12.832724 32745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:05:12.841325 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 32745
I20260927 07:05:12.841470 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:05:12.841929 13799 external_mini_cluster.cc:1428] Setting key d7e6d3f67b7877e969b9e473cc5e964b
I20260927 07:05:12.844787 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33551
--builtin_ntp_servers=127.13.121.212:32889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:05:12.845516 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:12.835342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:53325 (local address 127.13.121.254:33551)
0927 07:05:12.835557 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:12.835563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:12.836966 (+ 1403us) server_negotiation.cc:407] Connection header received
0927 07:05:12.838698 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:12.838703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:12.838772 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:12.838866 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:12.840075 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:12.840874 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:12.842599 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:12.843156 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:12.843285 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:12.844691 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:12.844696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:12.844699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:12.844716 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:12.844747 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:12.844760 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:12.844834 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:12.845122 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 07:05:12.845221 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 07:05:12.846463 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:12.835845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:33551 (local address 127.13.121.194:53325)
0927 07:05:12.836824 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:12.836859 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:12.838403 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:12.839175 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:12.839188 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:12.839289 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:12.839868 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:12.839891 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:12.841085 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:12.841090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:12.842446 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:12.842459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:12.843378 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:12.843383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:12.843634 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:12.844375 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:05:12.844377 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:12.844422 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:05:12.844505 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:12.844845 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:12.844855 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:12.844863 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:05:12.845076 (+ 213us) client_negotiation.cc:240] Negotiation successful
0927 07:05:12.845393 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":272,"threads_started":1}
I20260927 07:05:12.847918 404 heartbeater.cc:347] Connected to a master server at 127.13.121.254:33551
I20260927 07:05:12.848222 404 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:12.848935 404 heartbeater.cc:511] Master 127.13.121.254:33551 requested a full tablet report, sending...
I20260927 07:05:12.850356 32555 ts_manager.cc:194] Registered new tserver with Master: 7c15ebc5c63e4e9290c11651c20a4584 (127.13.121.194:42199)
I20260927 07:05:12.851253 32555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.194:53325
W20260927 07:05:13.172629 407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:13.172945 407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:13.173056 407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:13.181021 407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:13.181149 407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:13.181301 407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:05:13.190758 407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:32889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:13.192135 407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:13.193956 407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:13.208509 413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:13.208616 412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:13.208478 415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:13.209378 407 server_base.cc:1034] running on GCE node
I20260927 07:05:13.210096 407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:13.211159 407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:13.212402 407 hybrid_clock.cc:648] HybridClock initialized: now 1790492713212350 us; error 45 us; skew 500 ppm
I20260927 07:05:13.212755 407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:13.215245 407 webserver.cc:460] Webserver started at http://127.13.121.195:36111/ using document root <none> and password file <none>
I20260927 07:05:13.215978 407 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:13.216087 407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:13.216401 407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:13.219631 407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "415a2e595f4a42c4a733167633c18173"
format_stamp: "Formatted at 2026-09-27 07:05:13 on dist-test-slave-p6hz"
server_key: "be8a280de3bd509ea20adbddac0ed447"
server_key_iv: "e6bd64c27cbf50a91e92669675bc5d90"
server_key_version: "encryptionkey@0"
I20260927 07:05:13.220518 407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "415a2e595f4a42c4a733167633c18173"
format_stamp: "Formatted at 2026-09-27 07:05:13 on dist-test-slave-p6hz"
server_key: "be8a280de3bd509ea20adbddac0ed447"
server_key_iv: "e6bd64c27cbf50a91e92669675bc5d90"
server_key_version: "encryptionkey@0"
I20260927 07:05:13.225999 407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:05:13.229569 421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:13.231043 407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260927 07:05:13.231262 407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "415a2e595f4a42c4a733167633c18173"
format_stamp: "Formatted at 2026-09-27 07:05:13 on dist-test-slave-p6hz"
server_key: "be8a280de3bd509ea20adbddac0ed447"
server_key_iv: "e6bd64c27cbf50a91e92669675bc5d90"
server_key_version: "encryptionkey@0"
I20260927 07:05:13.231477 407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:13.254957 407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:13.255869 407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:13.256158 407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:13.257158 407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:13.258719 407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:13.258814 407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:13.258921 407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:13.258989 407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:13.289710 407 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:40437
I20260927 07:05:13.289783 533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:40437 every 8 connection(s)
I20260927 07:05:13.291177 407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:05:13.298988 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 407
I20260927 07:05:13.299176 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:05:13.299592 13799 external_mini_cluster.cc:1428] Setting key 94a00227c9977ab48820f1f78624fe6d
I20260927 07:05:13.304085 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.293792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:57281 (local address 127.13.121.254:33551)
0927 07:05:13.293983 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.293987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.295368 (+ 1381us) server_negotiation.cc:407] Connection header received
0927 07:05:13.297171 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.297176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.297250 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.297345 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:13.298381 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.299204 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.301052 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.301601 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.301720 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.303128 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:13.303165 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:13.303170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:13.303195 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:13.303233 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:13.303246 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:13.303322 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.303579 (+ 257us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.303685 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:05:13.305154 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.294320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:33551 (local address 127.13.121.195:57281)
0927 07:05:13.295231 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:13.295272 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:13.296876 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:13.297526 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:13.297534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:13.297630 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:13.298220 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.298236 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.299376 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.299381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.300909 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.300924 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.301839 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.301847 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.302092 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.302793 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:05:13.302795 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:13.302837 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:05:13.302927 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:13.303335 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:13.303347 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:13.303355 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:05:13.303553 (+ 198us) client_negotiation.cc:240] Negotiation successful
0927 07:05:13.304067 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":260,"threads_started":1}
I20260927 07:05:13.306561 534 heartbeater.cc:347] Connected to a master server at 127.13.121.254:33551
I20260927 07:05:13.306843 534 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:13.307619 534 heartbeater.cc:511] Master 127.13.121.254:33551 requested a full tablet report, sending...
I20260927 07:05:13.309038 32555 ts_manager.cc:194] Registered new tserver with Master: 415a2e595f4a42c4a733167633c18173 (127.13.121.195:40437)
I20260927 07:05:13.309792 32555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.195:57281
I20260927 07:05:13.314844 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:05:13.325222 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.318999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39918 (local address 127.13.121.254:33551)
0927 07:05:13.319242 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.319248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.319569 (+ 321us) server_negotiation.cc:407] Connection header received
0927 07:05:13.319754 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.319759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.319834 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.319919 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:13.320421 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.321203 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.322733 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.323197 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.323309 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.324502 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:13.324506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:13.324509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:13.324530 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:13.324563 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:13.324575 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:13.324698 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.324884 (+ 186us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.324973 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 07:05:13.329394 32555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:05:13.336306 32555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:05:13.376775 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.348940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42199 (local address 127.0.0.1:47324)
0927 07:05:13.362119 (+ 13179us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:13.362182 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:13.362422 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:13.369022 (+ 6600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:13.369027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:13.369079 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:13.369705 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.369719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.371337 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.371342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.373680 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.373700 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.375718 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.375725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.375940 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.376012 (+ 72us) client_negotiation.cc:769] Sending connection context
0927 07:05:13.376121 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 07:05:13.376343 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12360,"mutex_wait_us":20,"spinlock_wait_cycles":2816,"thread_start_us":18618,"threads_started":1}
I20260927 07:05:13.377151 546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.350693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:41627 (local address 127.0.0.1:59318)
0927 07:05:13.367866 (+ 17173us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:13.367886 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:13.368013 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:13.368978 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:13.368986 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:13.369020 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:13.369530 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.369551 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.372229 (+ 2678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.372234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.374189 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.374200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.376280 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.376285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.376417 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.376444 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 07:05:13.376523 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 07:05:13.376658 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17016,"thread_start_us":15218,"threads_started":1}
I20260927 07:05:13.377307 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.351295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59318 (local address 127.13.121.193:41627)
0927 07:05:13.368279 (+ 16984us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.368288 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.368325 (+ 37us) server_negotiation.cc:407] Connection header received
0927 07:05:13.368445 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.368452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.368739 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.368935 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:13.370066 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.372090 (+ 2024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.374356 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.375804 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.375978 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.376163 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.376609 (+ 446us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.376794 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16797,"thread_start_us":17191,"threads_started":1}
I20260927 07:05:13.378468 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.349108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.13.121.194:42199)
0927 07:05:13.368481 (+ 19373us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.368488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.368511 (+ 23us) server_negotiation.cc:407] Connection header received
0927 07:05:13.368585 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.368591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.368804 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.369480 (+ 676us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:13.369846 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.371204 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.373873 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.375495 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.377701 (+ 2206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.377910 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.377983 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.378159 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19226,"thread_start_us":19211,"threads_started":1}
I20260927 07:05:13.379755 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.349098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:40437 (local address 127.0.0.1:44176)
0927 07:05:13.366236 (+ 17138us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:13.366255 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:13.366385 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:13.369280 (+ 2895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:13.369285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:13.369311 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:13.369901 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.369915 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.371790 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.371795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.373472 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.373484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.376356 (+ 2872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.376362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.376533 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.376565 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:05:13.379288 (+ 2723us) client_negotiation.cc:240] Negotiation successful
0927 07:05:13.379434 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16990,"mutex_wait_us":19,"thread_start_us":12040,"threads_started":1}
I20260927 07:05:13.379887 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.349736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.13.121.195:40437)
0927 07:05:13.368419 (+ 18683us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.368427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.368460 (+ 33us) server_negotiation.cc:407] Connection header received
0927 07:05:13.368554 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.368562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.368805 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.368991 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:13.370118 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.371630 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.373659 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.375839 (+ 2180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.376015 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.376222 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.379346 (+ 3124us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.379544 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18513,"thread_start_us":18350,"threads_started":1}
I20260927 07:05:13.381187 32676 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7e9d5f069b4aeebbbfa7629fa15678 (DEFAULT_TABLE table=test-table [id=69d82a91aa484e7ab170a064d906a845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:13.382448 32676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f7e9d5f069b4aeebbbfa7629fa15678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:13.386456 469 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7e9d5f069b4aeebbbfa7629fa15678 (DEFAULT_TABLE table=test-table [id=69d82a91aa484e7ab170a064d906a845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:13.387888 469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f7e9d5f069b4aeebbbfa7629fa15678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:13.390480 339 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7e9d5f069b4aeebbbfa7629fa15678 (DEFAULT_TABLE table=test-table [id=69d82a91aa484e7ab170a064d906a845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:13.391970 339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f7e9d5f069b4aeebbbfa7629fa15678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:13.394366 552 tablet_bootstrap.cc:492] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d: Bootstrap starting.
I20260927 07:05:13.397780 552 tablet_bootstrap.cc:654] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:13.399176 552 log.cc:824] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:13.401759 552 tablet_bootstrap.cc:492] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d: No bootstrap required, opened a new log
I20260927 07:05:13.402061 552 ts_tablet_manager.cc:1397] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260927 07:05:13.402954 32741 heartbeater.cc:503] Master 127.13.121.254:33551 was elected leader, sending a full tablet report...
I20260927 07:05:13.404158 554 tablet_bootstrap.cc:492] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: Bootstrap starting.
I20260927 07:05:13.408771 555 tablet_bootstrap.cc:492] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: Bootstrap starting.
I20260927 07:05:13.409745 554 tablet_bootstrap.cc:654] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:13.411021 554 log.cc:824] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:13.412220 555 tablet_bootstrap.cc:654] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:13.412817 552 raft_consensus.cc:348] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } }
I20260927 07:05:13.413301 552 raft_consensus.cc:374] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:13.413332 555 log.cc:824] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:13.413404 552 raft_consensus.cc:729] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0d18af08dc4216b8ab57882568955d, State: Initialized, Role: FOLLOWER
I20260927 07:05:13.414063 552 consensus_queue.cc:260] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } }
I20260927 07:05:13.414526 554 tablet_bootstrap.cc:492] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: No bootstrap required, opened a new log
I20260927 07:05:13.414978 554 ts_tablet_manager.cc:1397] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260927 07:05:13.416788 552 ts_tablet_manager.cc:1428] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260927 07:05:13.417626 555 tablet_bootstrap.cc:492] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: No bootstrap required, opened a new log
I20260927 07:05:13.417943 555 ts_tablet_manager.cc:1397] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260927 07:05:13.424821 554 raft_consensus.cc:348] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } }
I20260927 07:05:13.425259 554 raft_consensus.cc:374] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:13.425364 554 raft_consensus.cc:729] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 415a2e595f4a42c4a733167633c18173, State: Initialized, Role: FOLLOWER
I20260927 07:05:13.425894 555 raft_consensus.cc:348] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } }
I20260927 07:05:13.425999 554 consensus_queue.cc:260] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } }
I20260927 07:05:13.426254 555 raft_consensus.cc:374] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:13.426348 555 raft_consensus.cc:729] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c15ebc5c63e4e9290c11651c20a4584, State: Initialized, Role: FOLLOWER
I20260927 07:05:13.427191 534 heartbeater.cc:503] Master 127.13.121.254:33551 was elected leader, sending a full tablet report...
I20260927 07:05:13.426994 555 consensus_queue.cc:260] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } }
I20260927 07:05:13.427881 554 ts_tablet_manager.cc:1428] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: Time spent starting tablet: real 0.013s user 0.007s sys 0.007s
I20260927 07:05:13.428191 404 heartbeater.cc:503] Master 127.13.121.254:33551 was elected leader, sending a full tablet report...
I20260927 07:05:13.430064 555 ts_tablet_manager.cc:1428] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
W20260927 07:05:13.542953 535 tablet.cc:2369] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:05:13.584844 405 tablet.cc:2369] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:05:13.591955 559 raft_consensus.cc:482] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:05:13.592249 559 raft_consensus.cc:504] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } }
I20260927 07:05:13.593982 559 leader_election.cc:288] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c15ebc5c63e4e9290c11651c20a4584 (127.13.121.194:42199), 5c0d18af08dc4216b8ab57882568955d (127.13.121.193:41627)
I20260927 07:05:13.599640 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.594804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:54241 (local address 127.13.121.194:42199)
0927 07:05:13.594998 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.595004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.595025 (+ 21us) server_negotiation.cc:407] Connection header received
0927 07:05:13.595115 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.595118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.595228 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.595313 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:13.595714 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.596400 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.598062 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.598783 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.598921 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.598992 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.599263 (+ 271us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.599386 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:05:13.599648 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.594314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:42199 (local address 127.13.121.195:54241)
0927 07:05:13.594481 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:13.594498 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:13.594636 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:13.595363 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:13.595368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:13.595394 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:13.595561 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.595570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.596543 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.596547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.597893 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.597906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.598921 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.598927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.599080 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.599104 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 07:05:13.599202 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 07:05:13.599337 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 07:05:13.600047 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.594904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:48841 (local address 127.13.121.193:41627)
0927 07:05:13.595115 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.595120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.595276 (+ 156us) server_negotiation.cc:407] Connection header received
0927 07:05:13.595434 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.595438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.595507 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.595580 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:13.596106 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.596773 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.598438 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.599222 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.599353 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.599420 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.599691 (+ 271us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.599813 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 07:05:13.600062 561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.594389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:41627 (local address 127.13.121.195:48841)
0927 07:05:13.595179 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:13.595195 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:13.595313 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:13.595621 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:13.595626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:13.595649 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:13.595941 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.595949 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.596913 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.596916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.598257 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:13.598269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.599370 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.599376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:13.599532 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.599562 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 07:05:13.599641 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 07:05:13.599759 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":424,"threads_started":1}
I20260927 07:05:13.600745 359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7e9d5f069b4aeebbbfa7629fa15678" candidate_uuid: "415a2e595f4a42c4a733167633c18173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c15ebc5c63e4e9290c11651c20a4584" is_pre_election: true
I20260927 07:05:13.601025 32696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7e9d5f069b4aeebbbfa7629fa15678" candidate_uuid: "415a2e595f4a42c4a733167633c18173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0d18af08dc4216b8ab57882568955d" is_pre_election: true
I20260927 07:05:13.601402 359 raft_consensus.cc:2454] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 415a2e595f4a42c4a733167633c18173 in term 0.
I20260927 07:05:13.601562 32696 raft_consensus.cc:2454] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 415a2e595f4a42c4a733167633c18173 in term 0.
I20260927 07:05:13.602108 422 leader_election.cc:302] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 415a2e595f4a42c4a733167633c18173, 7c15ebc5c63e4e9290c11651c20a4584; no voters:
I20260927 07:05:13.602483 559 raft_consensus.cc:2781] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:05:13.602618 559 raft_consensus.cc:482] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:05:13.602720 559 raft_consensus.cc:3037] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:13.605052 559 raft_consensus.cc:504] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } }
I20260927 07:05:13.605744 559 leader_election.cc:288] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [CANDIDATE]: Term 1 election: Requested vote from peers 7c15ebc5c63e4e9290c11651c20a4584 (127.13.121.194:42199), 5c0d18af08dc4216b8ab57882568955d (127.13.121.193:41627)
I20260927 07:05:13.606346 32696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7e9d5f069b4aeebbbfa7629fa15678" candidate_uuid: "415a2e595f4a42c4a733167633c18173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0d18af08dc4216b8ab57882568955d"
I20260927 07:05:13.606346 359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7e9d5f069b4aeebbbfa7629fa15678" candidate_uuid: "415a2e595f4a42c4a733167633c18173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c15ebc5c63e4e9290c11651c20a4584"
I20260927 07:05:13.606590 359 raft_consensus.cc:3037] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:13.606585 32696 raft_consensus.cc:3037] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:13.609877 32696 raft_consensus.cc:2454] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 415a2e595f4a42c4a733167633c18173 in term 1.
I20260927 07:05:13.609877 359 raft_consensus.cc:2454] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 415a2e595f4a42c4a733167633c18173 in term 1.
I20260927 07:05:13.610447 422 leader_election.cc:302] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 415a2e595f4a42c4a733167633c18173, 5c0d18af08dc4216b8ab57882568955d, 7c15ebc5c63e4e9290c11651c20a4584; no voters:
I20260927 07:05:13.610803 559 raft_consensus.cc:2781] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:13.611304 559 raft_consensus.cc:686] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 1 LEADER]: Becoming Leader. State: Replica: 415a2e595f4a42c4a733167633c18173, State: Running, Role: LEADER
I20260927 07:05:13.611792 559 consensus_queue.cc:237] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } }
I20260927 07:05:13.616695 32553 catalog_manager.cc:5495] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 reported cstate change: term changed from 0 to 1, leader changed from <none> to 415a2e595f4a42c4a733167633c18173 (127.13.121.195). New cstate: current_term: 1 leader_uuid: "415a2e595f4a42c4a733167633c18173" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "415a2e595f4a42c4a733167633c18173" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 40437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 } health_report { overall_health: UNKNOWN } } }
W20260927 07:05:13.631346 32742 tablet.cc:2369] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:05:13.677318 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.671769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.13.121.195:40437)
0927 07:05:13.671968 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.671974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.672057 (+ 83us) server_negotiation.cc:407] Connection header received
0927 07:05:13.672273 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.672278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.672366 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.672528 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:13.672904 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.673617 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.675009 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.675465 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.675575 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.676193 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:13.676720 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:13.676875 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.677037 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.677084 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 07:05:13.685848 32696 raft_consensus.cc:1261] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [term 1 FOLLOWER]: Refusing update from remote peer 415a2e595f4a42c4a733167633c18173: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:13.685854 359 raft_consensus.cc:1261] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [term 1 FOLLOWER]: Refusing update from remote peer 415a2e595f4a42c4a733167633c18173: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:13.686815 562 consensus_queue.cc:1035] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0d18af08dc4216b8ab57882568955d" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:13.687192 559 consensus_queue.cc:1035] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c15ebc5c63e4e9290c11651c20a4584" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 42199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:13.713124 32553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:13.713402 32553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:13.717396 32553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d329bbabff5a4725a11152b5d1cc600f: Sending DeleteTablet for 3 replicas of tablet 3f7e9d5f069b4aeebbbfa7629fa15678
I20260927 07:05:13.718688 339 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7e9d5f069b4aeebbbfa7629fa15678 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:13 UTC) from {username='slave'} at 127.0.0.1:47324
I20260927 07:05:13.718894 32676 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7e9d5f069b4aeebbbfa7629fa15678 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:13 UTC) from {username='slave'} at 127.0.0.1:59318
I20260927 07:05:13.719642 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 32614
I20260927 07:05:13.722124 469 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7e9d5f069b4aeebbbfa7629fa15678 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:13 UTC) from {username='slave'} at 127.0.0.1:44176
I20260927 07:05:13.746215 577 tablet_replica.cc:330] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: stopping tablet replica
I20260927 07:05:13.747074 577 raft_consensus.cc:2229] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:05:13.749155 575 tablet_replica.cc:330] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d: stopping tablet replica
I20260927 07:05:13.749550 577 raft_consensus.cc:2258] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:13.750118 575 raft_consensus.cc:2229] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:13.751035 575 raft_consensus.cc:2258] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:13.753278 575 ts_tablet_manager.cc:1905] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 5c0d18af08dc4216b8ab57882568955d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:13.753279 577 ts_tablet_manager.cc:1905] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:13.756369 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.720571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59326 (local address 127.13.121.193:41627)
0927 07:05:13.720774 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.720780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.748016 (+ 27236us) server_negotiation.cc:407] Connection header received
0927 07:05:13.748107 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.748113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.748232 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.748330 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:13.749016 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.750185 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.752250 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.752868 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.752970 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.754542 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:13.754554 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:13.754583 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:13.755412 (+ 829us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:13.755492 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:13.755526 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:13.755631 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.755881 (+ 250us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.756000 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 07:05:13.757362 574 tablet_replica.cc:330] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: stopping tablet replica
I20260927 07:05:13.757628 32736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:13.758080 574 raft_consensus.cc:2229] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:13.760568 574 raft_consensus.cc:2258] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:13.761125 577 ts_tablet_manager.cc:1918] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:13.761319 577 log.cc:1196] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/3f7e9d5f069b4aeebbbfa7629fa15678
I20260927 07:05:13.762054 577 ts_tablet_manager.cc:1939] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 415a2e595f4a42c4a733167633c18173: Deleting consensus metadata
I20260927 07:05:13.763345 574 ts_tablet_manager.cc:1905] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:13.764151 32540 catalog_manager.cc:4841] TS 415a2e595f4a42c4a733167633c18173 (127.13.121.195:40437): tablet 3f7e9d5f069b4aeebbbfa7629fa15678 (table test-table [id=69d82a91aa484e7ab170a064d906a845]) successfully deleted
I20260927 07:05:13.770874 574 ts_tablet_manager.cc:1918] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:13.771037 574 log.cc:1196] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/3f7e9d5f069b4aeebbbfa7629fa15678
I20260927 07:05:13.771526 574 ts_tablet_manager.cc:1939] T 3f7e9d5f069b4aeebbbfa7629fa15678 P 7c15ebc5c63e4e9290c11651c20a4584: Deleting consensus metadata
I20260927 07:05:13.773379 32539 catalog_manager.cc:4841] TS 7c15ebc5c63e4e9290c11651c20a4584 (127.13.121.194:42199): tablet 3f7e9d5f069b4aeebbbfa7629fa15678 (table test-table [id=69d82a91aa484e7ab170a064d906a845]) successfully deleted
I20260927 07:05:13.868239 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 32614
I20260927 07:05:13.890475 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 32745
W20260927 07:05:13.890596 32541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.121.193:41627 (error 108)
I20260927 07:05:13.894776 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.892572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:41627 (local address 127.0.0.1:59330)
0927 07:05:13.894478 (+ 1906us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:13.894612 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:41627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1789,"thread_start_us":1695,"threads_started":1}
W20260927 07:05:13.895059 32541 catalog_manager.cc:4584] TS 5c0d18af08dc4216b8ab57882568955d (127.13.121.193:41627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f7e9d5f069b4aeebbbfa7629fa15678: Network error: Client connection negotiation failed: client connection to 127.13.121.193:41627: connect: Connection refused (error 111)
I20260927 07:05:13.898805 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.891400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.13.121.194:42199)
0927 07:05:13.891616 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:13.891622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:13.891646 (+ 24us) server_negotiation.cc:407] Connection header received
0927 07:05:13.891863 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:13.891869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:13.891978 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:13.892090 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:13.892719 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.893991 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.895842 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:13.896279 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:13.896379 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:13.897469 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:13.897477 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:13.897492 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:13.898072 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:13.898128 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:13.898150 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:13.898237 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:13.898382 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 07:05:13.898487 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:05:13.899556 399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:13.948728 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.948170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:41627 (local address 127.0.0.1:59342)
0927 07:05:13.948428 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:13.948551 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:41627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260927 07:05:13.949918 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:13.949472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:41627 (local address 127.0.0.1:59352)
0927 07:05:13.949650 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:13.949755 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:41627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 07:05:13.989279 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 32745
I20260927 07:05:14.003471 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 407
I20260927 07:05:14.010633 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:14.010124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:41627 (local address 127.0.0.1:59354)
0927 07:05:14.010344 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:14.010451 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:41627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260927 07:05:14.011672 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:14.004211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44190 (local address 127.13.121.195:40437)
0927 07:05:14.004365 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:14.004370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:14.004460 (+ 90us) server_negotiation.cc:407] Connection header received
0927 07:05:14.004634 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:14.004638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:14.004719 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:14.004793 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:14.005273 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:14.006086 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:14.008057 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:14.008472 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:14.008634 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:14.009739 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:14.009750 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:14.009769 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:14.010529 (+ 760us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:14.010585 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:14.010612 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:14.010704 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:14.011170 (+ 466us) server_negotiation.cc:299] Negotiation successful
0927 07:05:14.011287 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 07:05:14.011759 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:14.011321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:41627 (local address 127.0.0.1:59370)
0927 07:05:14.011489 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:14.011581 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:41627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 07:05:14.012414 529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:14.092542 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:14.091975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:41627 (local address 127.0.0.1:59380)
0927 07:05:14.092253 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:14.092367 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:41627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260927 07:05:14.093742 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:14.093265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:41627 (local address 127.0.0.1:59386)
0927 07:05:14.093469 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:14.093582 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.193:41627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260927 07:05:14.134552 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 407
I20260927 07:05:14.150115 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 32524
I20260927 07:05:14.151369 32585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:14.259239 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 32524
2026-09-27T07:05:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2955 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:14 dist-test-slave-p6hz krb5kdc[586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:14 dist-test-slave-p6hz krb5kdc[586](info): set up 2 sockets
Sep 27 07:05:14 dist-test-slave-p6hz krb5kdc[586](info): commencing operation
krb5kdc: starting...
W20260927 07:05:15.283430 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:15 dist-test-slave-p6hz krb5kdc[586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:15.531265 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:33113
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:33113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:15.862599 599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:15.862927 599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:15.863065 599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:15.870556 599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:15.870684 599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:15.870733 599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:15.870812 599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:15.870873 599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:15.881237 599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:33113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:33113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:15.882679 599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:15.884060 599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:15.895826 604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:15.895768 607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:15.895805 605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:15.897068 599 server_base.cc:1034] running on GCE node
I20260927 07:05:15.897882 599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:15.899397 599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:15.900635 599 hybrid_clock.cc:648] HybridClock initialized: now 1790492715900569 us; error 52 us; skew 500 ppm
I20260927 07:05:15.901038 599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:15 dist-test-slave-p6hz krb5kdc[586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492715, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:15.909848 599 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:15.911573 599 webserver.cc:460] Webserver started at http://127.13.121.254:35177/ using document root <none> and password file <none>
I20260927 07:05:15.912309 599 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:15.912436 599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:15.912742 599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:15.915994 599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "30a8bcc876d4442a978e4a89bd00f889"
format_stamp: "Formatted at 2026-09-27 07:05:15 on dist-test-slave-p6hz"
server_key: "0516cf1fd9c32d314fe07ccb831e94a7"
server_key_iv: "ca150784806186c3cf196d656d0d266e"
server_key_version: "encryptionkey@0"
I20260927 07:05:15.916827 599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "30a8bcc876d4442a978e4a89bd00f889"
format_stamp: "Formatted at 2026-09-27 07:05:15 on dist-test-slave-p6hz"
server_key: "0516cf1fd9c32d314fe07ccb831e94a7"
server_key_iv: "ca150784806186c3cf196d656d0d266e"
server_key_version: "encryptionkey@0"
I20260927 07:05:15.922328 599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260927 07:05:15.926028 614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:15.927484 599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:05:15.927655 599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "30a8bcc876d4442a978e4a89bd00f889"
format_stamp: "Formatted at 2026-09-27 07:05:15 on dist-test-slave-p6hz"
server_key: "0516cf1fd9c32d314fe07ccb831e94a7"
server_key_iv: "ca150784806186c3cf196d656d0d266e"
server_key_version: "encryptionkey@0"
I20260927 07:05:15.927845 599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:15.940613 599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:15.945356 599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:15.945664 599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:15.963721 599 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:33113
I20260927 07:05:15.963752 665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:33113 every 8 connection(s)
I20260927 07:05:15.965238 599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:15.969282 666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:15.975579 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 599
I20260927 07:05:15.975759 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:15.976179 13799 external_mini_cluster.cc:1428] Setting key 2f3ce535f3e9071b65ca56e1a934be8d
I20260927 07:05:15.980577 666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889: Bootstrap starting.
I20260927 07:05:15.984534 666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:15.985735 666 log.cc:824] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:15.989331 666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889: No bootstrap required, opened a new log
Sep 27 07:05:15 dist-test-slave-p6hz krb5kdc[586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:15.997726 666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a8bcc876d4442a978e4a89bd00f889" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33113 } }
I20260927 07:05:15.998116 666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:15.998232 666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a8bcc876d4442a978e4a89bd00f889, State: Initialized, Role: FOLLOWER
I20260927 07:05:15.998860 666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a8bcc876d4442a978e4a89bd00f889" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33113 } }
I20260927 07:05:15.999109 666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:15.999276 666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:15.999460 666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:16.000625 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:15.978144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52018 (local address 127.13.121.254:33113)
0927 07:05:15.978999 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:15.979012 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:15.979056 (+ 44us) server_negotiation.cc:407] Connection header received
0927 07:05:15.980958 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:15.980984 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:15.981446 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:15.981801 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:15.983804 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:15.985253 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:15.986958 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:15.987785 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:15.987941 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:15.992850 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:15.992859 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:15.992878 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:15.992933 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:15.996199 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:15.996982 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:15.996992 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:15.997000 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:15.997190 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:15.997611 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:15.997615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:15.997617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:15.998266 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:15.998578 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:15.999019 (+ 441us) server_negotiation.cc:299] Negotiation successful
0927 07:05:15.999522 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":408,"threads_started":1}
I20260927 07:05:16.001885 666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a8bcc876d4442a978e4a89bd00f889" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33113 } }
I20260927 07:05:16.002417 666 leader_election.cc:302] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30a8bcc876d4442a978e4a89bd00f889; no voters:
I20260927 07:05:16.003093 666 leader_election.cc:288] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:16.003298 671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:16.003993 671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [term 1 LEADER]: Becoming Leader. State: Replica: 30a8bcc876d4442a978e4a89bd00f889, State: Running, Role: LEADER
I20260927 07:05:16.004609 671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a8bcc876d4442a978e4a89bd00f889" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33113 } }
I20260927 07:05:16.005532 666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:16.007395 673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30a8bcc876d4442a978e4a89bd00f889. Latest consensus state: current_term: 1 leader_uuid: "30a8bcc876d4442a978e4a89bd00f889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a8bcc876d4442a978e4a89bd00f889" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33113 } } }
I20260927 07:05:16.007866 673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:16.008406 672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30a8bcc876d4442a978e4a89bd00f889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a8bcc876d4442a978e4a89bd00f889" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33113 } } }
I20260927 07:05:16.008703 672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:16.011291 680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:16.015925 680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:16.025027 680 catalog_manager.cc:1347] Generated new cluster ID: 066c0488655647f4a584bea67b66bb9b
I20260927 07:05:16.025163 680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:16.054201 680 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:16.055685 680 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:16.067451 680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889: Generated new TSK 0
I20260927 07:05:16.068379 680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.121.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
Entry for principal kudu/127.13.121.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab.
I20260927 07:05:16.137229 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33113
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:16.469906 692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:16.470233 692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:16.470361 692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:16.477807 692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:16.477948 692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:16.478108 692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:05:16.488256 692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.193.keytab
--principal=kudu/127.13.121.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:16.489595 692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:16.490934 692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:16.503997 700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:16.503991 698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:16.504583 697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:16.504639 692 server_base.cc:1034] running on GCE node
I20260927 07:05:16.505324 692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:16.506341 692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:16.507555 692 hybrid_clock.cc:648] HybridClock initialized: now 1790492716507482 us; error 64 us; skew 500 ppm
I20260927 07:05:16.507912 692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:16 dist-test-slave-p6hz krb5kdc[586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492716, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:16.511073 692 init.cc:377] Logged in from keytab as kudu/127.13.121.193@KRBTEST.COM (short username kudu)
I20260927 07:05:16.512667 692 webserver.cc:460] Webserver started at http://127.13.121.193:44367/ using document root <none> and password file <none>
I20260927 07:05:16.513363 692 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:16.513489 692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:16.513799 692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:16.516804 692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "4043dbfcf1004ff489a44e528eec6c9a"
format_stamp: "Formatted at 2026-09-27 07:05:16 on dist-test-slave-p6hz"
server_key: "916bac68c2bb48ced9c1c90097eea8a9"
server_key_iv: "b717abe1ecab32f05a3b68fbfe89de23"
server_key_version: "encryptionkey@0"
I20260927 07:05:16.517534 692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "4043dbfcf1004ff489a44e528eec6c9a"
format_stamp: "Formatted at 2026-09-27 07:05:16 on dist-test-slave-p6hz"
server_key: "916bac68c2bb48ced9c1c90097eea8a9"
server_key_iv: "b717abe1ecab32f05a3b68fbfe89de23"
server_key_version: "encryptionkey@0"
I20260927 07:05:16.523036 692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 07:05:16.526674 707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:16.528108 692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 07:05:16.528272 692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "4043dbfcf1004ff489a44e528eec6c9a"
format_stamp: "Formatted at 2026-09-27 07:05:16 on dist-test-slave-p6hz"
server_key: "916bac68c2bb48ced9c1c90097eea8a9"
server_key_iv: "b717abe1ecab32f05a3b68fbfe89de23"
server_key_version: "encryptionkey@0"
I20260927 07:05:16.528468 692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:16.552692 692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:16.557133 692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:16.557432 692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:16.558451 692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:16.559876 692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:16.559958 692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:16.560081 692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:16.560132 692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:16.591802 692 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:36877
I20260927 07:05:16.591822 826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:36877 every 8 connection(s)
I20260927 07:05:16.593135 692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:05:16.594740 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 692
I20260927 07:05:16.594884 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:05:16.595376 13799 external_mini_cluster.cc:1428] Setting key bb418642e89162e4f3ebe32abdc48283
Sep 27 07:05:16 dist-test-slave-p6hz krb5kdc[586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492716, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.193@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:16.615916 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:16.595629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:57633 (local address 127.13.121.254:33113)
0927 07:05:16.596109 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:16.596115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:16.597433 (+ 1318us) server_negotiation.cc:407] Connection header received
0927 07:05:16.599631 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:16.599636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:16.599704 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:16.599799 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:16.601713 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:16.602444 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:16.604641 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:16.605102 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:16.605211 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:16.609097 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:16.609105 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:16.609109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:16.609151 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:16.612222 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:16.613156 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:16.613163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:16.613167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:16.613324 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:16.613827 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:16.613832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:16.613836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:16.614230 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:16.614379 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:16.615236 (+ 857us) server_negotiation.cc:299] Negotiation successful
0927 07:05:16.615471 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":208,"threads_started":1}
I20260927 07:05:16.616693 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:16.596193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:33113 (local address 127.13.121.193:57633)
0927 07:05:16.596976 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:16.597066 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:16.599335 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:16.600052 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:16.600064 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:16.600775 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:16.601523 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:16.601548 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:16.602596 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:16.602605 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:16.604479 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:16.604494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:16.605329 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:16.605336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:16.605603 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:16.606369 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:16.606400 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:16.608802 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:16.612403 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:16.612411 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:16.612426 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:16.612917 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:16.613469 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:16.613472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:16.613475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:16.613706 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:16.614386 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:16.614395 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:16.614798 (+ 403us) client_negotiation.cc:769] Sending connection context
0927 07:05:16.615053 (+ 255us) client_negotiation.cc:240] Negotiation successful
0927 07:05:16.615547 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":338,"threads_started":1}
I20260927 07:05:16.618340 827 heartbeater.cc:347] Connected to a master server at 127.13.121.254:33113
I20260927 07:05:16.618727 827 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:16.619719 827 heartbeater.cc:511] Master 127.13.121.254:33113 requested a full tablet report, sending...
I20260927 07:05:16.622135 631 ts_manager.cc:194] Registered new tserver with Master: 4043dbfcf1004ff489a44e528eec6c9a (127.13.121.193:36877)
WARNING: no policy specified for kudu/127.13.121.194@KRBTEST.COM; defaulting to no policy
I20260927 07:05:16.623786 631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.193@KRBTEST.COM'} at 127.13.121.193:57633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
Entry for principal kudu/127.13.121.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab.
I20260927 07:05:16.654321 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33113
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:16.975929 833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:16.976205 833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:16.976344 833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:16.983696 833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:16.983836 833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:16.983985 833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:05:16.993996 833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.194.keytab
--principal=kudu/127.13.121.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:16.995419 833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:16.996855 833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:17.009721 838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:17.009805 841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:17.009768 839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:17.010535 833 server_base.cc:1034] running on GCE node
I20260927 07:05:17.011232 833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:17.012193 833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:17.013413 833 hybrid_clock.cc:648] HybridClock initialized: now 1790492717013358 us; error 48 us; skew 500 ppm
I20260927 07:05:17.013734 833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:17 dist-test-slave-p6hz krb5kdc[586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492717, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:17.016919 833 init.cc:377] Logged in from keytab as kudu/127.13.121.194@KRBTEST.COM (short username kudu)
I20260927 07:05:17.018450 833 webserver.cc:460] Webserver started at http://127.13.121.194:42221/ using document root <none> and password file <none>
I20260927 07:05:17.019157 833 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:17.019292 833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:17.019603 833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:17.022588 833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "847a6e6dd7944d8ebd9f673fe12ab746"
format_stamp: "Formatted at 2026-09-27 07:05:17 on dist-test-slave-p6hz"
server_key: "1ecaffe3ad75473ea5bc2723c8bb5a24"
server_key_iv: "b3eb2fbfee8ef49cfe927bfe597eeb88"
server_key_version: "encryptionkey@0"
I20260927 07:05:17.023447 833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "847a6e6dd7944d8ebd9f673fe12ab746"
format_stamp: "Formatted at 2026-09-27 07:05:17 on dist-test-slave-p6hz"
server_key: "1ecaffe3ad75473ea5bc2723c8bb5a24"
server_key_iv: "b3eb2fbfee8ef49cfe927bfe597eeb88"
server_key_version: "encryptionkey@0"
I20260927 07:05:17.028798 833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:05:17.032548 848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:17.034008 833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:17.034206 833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "847a6e6dd7944d8ebd9f673fe12ab746"
format_stamp: "Formatted at 2026-09-27 07:05:17 on dist-test-slave-p6hz"
server_key: "1ecaffe3ad75473ea5bc2723c8bb5a24"
server_key_iv: "b3eb2fbfee8ef49cfe927bfe597eeb88"
server_key_version: "encryptionkey@0"
I20260927 07:05:17.034395 833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:17.048091 833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:17.052196 833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:17.052465 833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:17.053406 833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:17.054713 833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:17.054788 833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:17.054905 833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:17.054960 833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:17.085512 833 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:39447
I20260927 07:05:17.085562 960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:39447 every 8 connection(s)
I20260927 07:05:17.086789 833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:05:17.087996 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 833
I20260927 07:05:17.088172 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:05:17.088575 13799 external_mini_cluster.cc:1428] Setting key 34e0d5c9875f6d148f960d09e291700e
Sep 27 07:05:17 dist-test-slave-p6hz krb5kdc[586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492717, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.194@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:17.108121 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.089435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:34723 (local address 127.13.121.254:33113)
0927 07:05:17.089633 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.089640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.090903 (+ 1263us) server_negotiation.cc:407] Connection header received
0927 07:05:17.092739 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.092744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.092824 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.092910 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:17.094491 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.095267 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.096911 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.097342 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.097457 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.101622 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:17.101629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:17.101632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:17.101679 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:17.104322 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:17.105383 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:17.105391 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:17.105394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:17.105579 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:17.106114 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:17.106118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:17.106120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:17.106431 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:17.106606 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.107495 (+ 889us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.107771 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:05:17.109222 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.089940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:33113 (local address 127.13.121.194:34723)
0927 07:05:17.090765 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:17.090812 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:17.092440 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:17.093182 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:17.093192 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:17.093735 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:17.094308 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.094327 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.095418 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.095422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.096762 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.096775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.097584 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.097592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.097878 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.098634 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:17.098663 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:17.101323 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:17.104565 (+ 3242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:17.104577 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:17.104599 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:17.105198 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:17.105705 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:17.105710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:17.105713 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:17.105980 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:17.106617 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:17.106627 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:17.106994 (+ 367us) client_negotiation.cc:769] Sending connection context
0927 07:05:17.107371 (+ 377us) client_negotiation.cc:240] Negotiation successful
0927 07:05:17.107964 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":294,"threads_started":1}
I20260927 07:05:17.110764 961 heartbeater.cc:347] Connected to a master server at 127.13.121.254:33113
I20260927 07:05:17.111166 961 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:17.112094 961 heartbeater.cc:511] Master 127.13.121.254:33113 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.121.195@KRBTEST.COM; defaulting to no policy
I20260927 07:05:17.113790 631 ts_manager.cc:194] Registered new tserver with Master: 847a6e6dd7944d8ebd9f673fe12ab746 (127.13.121.194:39447)
I20260927 07:05:17.114683 631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.194@KRBTEST.COM'} at 127.13.121.194:34723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
Entry for principal kudu/127.13.121.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab.
I20260927 07:05:17.143208 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33113
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:17.461401 966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:17.461714 966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:17.461808 966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:17.469636 966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:17.469765 966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:17.469885 966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:05:17.479779 966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.195.keytab
--principal=kudu/127.13.121.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:33113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:17.481055 966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:17.482277 966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:17.495455 971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:17.495492 972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:17.495779 974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:17.496745 966 server_base.cc:1034] running on GCE node
I20260927 07:05:17.497385 966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:17.498368 966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:17.499554 966 hybrid_clock.cc:648] HybridClock initialized: now 1790492717499495 us; error 58 us; skew 500 ppm
I20260927 07:05:17.499877 966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:17 dist-test-slave-p6hz krb5kdc[586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492717, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:17.502928 966 init.cc:377] Logged in from keytab as kudu/127.13.121.195@KRBTEST.COM (short username kudu)
I20260927 07:05:17.504547 966 webserver.cc:460] Webserver started at http://127.13.121.195:44209/ using document root <none> and password file <none>
I20260927 07:05:17.505266 966 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:17.505409 966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:17.505740 966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:17.508913 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "0bcf1b0109e749ae97b6be72a898c62e"
format_stamp: "Formatted at 2026-09-27 07:05:17 on dist-test-slave-p6hz"
server_key: "5a1807fd237fb30f6fbcf227d2057b74"
server_key_iv: "bf652b5367855e2109ab9415f7bb5e8f"
server_key_version: "encryptionkey@0"
I20260927 07:05:17.509732 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "0bcf1b0109e749ae97b6be72a898c62e"
format_stamp: "Formatted at 2026-09-27 07:05:17 on dist-test-slave-p6hz"
server_key: "5a1807fd237fb30f6fbcf227d2057b74"
server_key_iv: "bf652b5367855e2109ab9415f7bb5e8f"
server_key_version: "encryptionkey@0"
I20260927 07:05:17.515285 966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260927 07:05:17.519344 981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:17.520980 966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:17.521181 966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "0bcf1b0109e749ae97b6be72a898c62e"
format_stamp: "Formatted at 2026-09-27 07:05:17 on dist-test-slave-p6hz"
server_key: "5a1807fd237fb30f6fbcf227d2057b74"
server_key_iv: "bf652b5367855e2109ab9415f7bb5e8f"
server_key_version: "encryptionkey@0"
I20260927 07:05:17.521413 966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:17.538729 966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:17.543494 966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:17.543783 966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:17.544854 966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:17.546275 966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:17.546355 966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:17.546481 966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:17.546535 966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:17.578159 966 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:33677
I20260927 07:05:17.578182 1099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:33677 every 8 connection(s)
I20260927 07:05:17.579475 966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:05:17.586515 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 966
I20260927 07:05:17.586710 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:05:17.587102 13799 external_mini_cluster.cc:1428] Setting key 70322dd7095599254596d80df82f515e
Sep 27 07:05:17 dist-test-slave-p6hz krb5kdc[586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492717, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.195@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:17.600823 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.582077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:53377 (local address 127.13.121.254:33113)
0927 07:05:17.582284 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.582288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.583459 (+ 1171us) server_negotiation.cc:407] Connection header received
0927 07:05:17.585282 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.585289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.585388 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.585474 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:17.587065 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.588074 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.590096 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.590539 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.590647 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.594363 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:17.594372 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:17.594376 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:17.594423 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:17.597550 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:17.598410 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:17.598415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:17.598418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:17.598582 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:17.599020 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:17.599056 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:17.599058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:17.599413 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:17.599541 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.600277 (+ 736us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.600499 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260927 07:05:17.602078 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.582497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:33113 (local address 127.13.121.195:53377)
0927 07:05:17.583294 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:17.583336 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:17.584992 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:17.585727 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:17.585736 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:17.586279 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:17.586888 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.586909 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.588261 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.588267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.589931 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.589947 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.590776 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.590782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.591013 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.591739 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:17.591763 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:17.594121 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:17.597731 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:17.597739 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:17.597753 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:17.598265 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:17.598697 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:17.598701 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:17.598703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:17.598900 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:17.599543 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:17.599552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:17.599901 (+ 349us) client_negotiation.cc:769] Sending connection context
0927 07:05:17.600196 (+ 295us) client_negotiation.cc:240] Negotiation successful
0927 07:05:17.600685 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":263,"threads_started":1}
I20260927 07:05:17.603387 1100 heartbeater.cc:347] Connected to a master server at 127.13.121.254:33113
I20260927 07:05:17.603672 1100 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:17.604358 1100 heartbeater.cc:511] Master 127.13.121.254:33113 requested a full tablet report, sending...
I20260927 07:05:17.605787 631 ts_manager.cc:194] Registered new tserver with Master: 0bcf1b0109e749ae97b6be72a898c62e (127.13.121.195:33677)
I20260927 07:05:17.606628 631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.121.195@KRBTEST.COM'} at 127.13.121.195:53377
I20260927 07:05:17.614815 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:05:17.627070 827 heartbeater.cc:503] Master 127.13.121.254:33113 was elected leader, sending a full tablet report...
I20260927 07:05:17.632575 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.618824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57072 (local address 127.13.121.254:33113)
0927 07:05:17.619043 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.619051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.619388 (+ 337us) server_negotiation.cc:407] Connection header received
0927 07:05:17.619543 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.619547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.619615 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.619704 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:17.621105 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.621878 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.623326 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.623786 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.623897 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.626469 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:17.626475 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:17.626479 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:17.626520 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:17.629649 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:17.630381 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:17.630386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:17.630388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:17.630545 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:17.630923 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:17.630946 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:17.630949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:17.631357 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:17.631650 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.631984 (+ 334us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.632221 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:05:17.636490 631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:05:17.643486 631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:05:17.689436 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.658842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37872 (local address 127.13.121.193:36877)
0927 07:05:17.678777 (+ 19935us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.678790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.681660 (+ 2870us) server_negotiation.cc:407] Connection header received
0927 07:05:17.681957 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.681965 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.682153 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.682322 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:17.683157 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.684321 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.686593 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.687863 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.688121 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.688308 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.688932 (+ 624us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.689117 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19683,"thread_start_us":19959,"threads_started":1}
I20260927 07:05:17.689544 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.658375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:36877 (local address 127.0.0.1:37872)
0927 07:05:17.681516 (+ 23141us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:17.681538 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:17.681769 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:17.682605 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:17.682613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:17.682639 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:17.682956 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.682965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.684480 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.684484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.686417 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.686430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.688457 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.688464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.688664 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.688722 (+ 58us) client_negotiation.cc:769] Sending connection context
0927 07:05:17.688853 (+ 131us) client_negotiation.cc:240] Negotiation successful
0927 07:05:17.689062 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23002,"mutex_wait_us":69,"thread_start_us":22561,"threads_started":1}
I20260927 07:05:17.690073 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.656121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:39447 (local address 127.0.0.1:46540)
0927 07:05:17.680568 (+ 24447us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:17.680603 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:17.680786 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:17.681944 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:17.681952 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:17.681981 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:17.682406 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.682419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.684340 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.684346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.686989 (+ 2643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.687003 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.688967 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.688973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.689137 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.689169 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:05:17.689256 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 07:05:17.689385 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24319,"mutex_wait_us":213,"thread_start_us":25092,"threads_started":1}
I20260927 07:05:17.690346 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.656203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46540 (local address 127.13.121.194:39447)
0927 07:05:17.681123 (+ 24920us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.681131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.681161 (+ 30us) server_negotiation.cc:407] Connection header received
0927 07:05:17.681265 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.681273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.681552 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.681747 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:17.682572 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.684172 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.687189 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.688536 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.688686 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.688833 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.689865 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.690044 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24775,"thread_start_us":24660,"threads_started":1}
I20260927 07:05:17.690980 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.655836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33677 (local address 127.0.0.1:35104)
0927 07:05:17.680443 (+ 24607us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:17.680517 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:17.681449 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:17.682170 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:17.682174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:17.682199 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:17.682672 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.682688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.685060 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.685065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.687295 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.687305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.689364 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.689370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.689545 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.690428 (+ 883us) client_negotiation.cc:769] Sending connection context
0927 07:05:17.690506 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 07:05:17.690646 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24268,"mutex_wait_us":23,"thread_start_us":24095,"threads_started":1}
I20260927 07:05:17.691290 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.656059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35104 (local address 127.13.121.195:33677)
0927 07:05:17.679559 (+ 23500us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.679568 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.680691 (+ 1123us) server_negotiation.cc:407] Connection header received
0927 07:05:17.681676 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.681685 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.681920 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.682106 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:17.682888 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.684899 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.687435 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.688881 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.689054 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.689208 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.690770 (+ 1562us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.690932 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23301,"thread_start_us":23464,"threads_started":1}
I20260927 07:05:17.693874 896 tablet_service.cc:1467] Processing CreateTablet for tablet b693c1232ae345238efbe716f1061189 (DEFAULT_TABLE table=test-table [id=04e026f758e94f10b552a1377b7c1b09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:17.693871 755 tablet_service.cc:1467] Processing CreateTablet for tablet b693c1232ae345238efbe716f1061189 (DEFAULT_TABLE table=test-table [id=04e026f758e94f10b552a1377b7c1b09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:17.695336 896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b693c1232ae345238efbe716f1061189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:17.695358 755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b693c1232ae345238efbe716f1061189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:17.696460 1029 tablet_service.cc:1467] Processing CreateTablet for tablet b693c1232ae345238efbe716f1061189 (DEFAULT_TABLE table=test-table [id=04e026f758e94f10b552a1377b7c1b09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:17.697800 1029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b693c1232ae345238efbe716f1061189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:17.707345 1118 tablet_bootstrap.cc:492] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: Bootstrap starting.
I20260927 07:05:17.710608 1118 tablet_bootstrap.cc:654] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:17.711844 1118 log.cc:824] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:17.711898 1119 tablet_bootstrap.cc:492] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: Bootstrap starting.
I20260927 07:05:17.715902 1119 tablet_bootstrap.cc:654] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:17.717411 1119 log.cc:824] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:17.718887 1120 tablet_bootstrap.cc:492] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: Bootstrap starting.
I20260927 07:05:17.720877 1118 tablet_bootstrap.cc:492] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: No bootstrap required, opened a new log
I20260927 07:05:17.720877 1119 tablet_bootstrap.cc:492] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: No bootstrap required, opened a new log
I20260927 07:05:17.721166 1119 ts_tablet_manager.cc:1397] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260927 07:05:17.721166 1118 ts_tablet_manager.cc:1397] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260927 07:05:17.721916 1120 tablet_bootstrap.cc:654] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:17.724241 1120 log.cc:824] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:17.727026 1120 tablet_bootstrap.cc:492] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: No bootstrap required, opened a new log
I20260927 07:05:17.727309 1120 ts_tablet_manager.cc:1397] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260927 07:05:17.731671 1119 raft_consensus.cc:348] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } }
I20260927 07:05:17.731684 1118 raft_consensus.cc:348] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } }
I20260927 07:05:17.732149 1118 raft_consensus.cc:374] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:17.732149 1119 raft_consensus.cc:374] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:17.732246 1119 raft_consensus.cc:729] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4043dbfcf1004ff489a44e528eec6c9a, State: Initialized, Role: FOLLOWER
I20260927 07:05:17.732285 1118 raft_consensus.cc:729] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847a6e6dd7944d8ebd9f673fe12ab746, State: Initialized, Role: FOLLOWER
I20260927 07:05:17.732926 1119 consensus_queue.cc:260] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } }
I20260927 07:05:17.732926 1118 consensus_queue.cc:260] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } }
I20260927 07:05:17.735445 1120 raft_consensus.cc:348] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } }
I20260927 07:05:17.735965 1120 raft_consensus.cc:374] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:17.736114 1120 raft_consensus.cc:729] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bcf1b0109e749ae97b6be72a898c62e, State: Initialized, Role: FOLLOWER
I20260927 07:05:17.736388 961 heartbeater.cc:503] Master 127.13.121.254:33113 was elected leader, sending a full tablet report...
I20260927 07:05:17.736774 1120 consensus_queue.cc:260] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } }
I20260927 07:05:17.738121 1119 ts_tablet_manager.cc:1428] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260927 07:05:17.738238 1118 ts_tablet_manager.cc:1428] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: Time spent starting tablet: real 0.017s user 0.011s sys 0.007s
I20260927 07:05:17.739094 1100 heartbeater.cc:503] Master 127.13.121.254:33113 was elected leader, sending a full tablet report...
I20260927 07:05:17.739661 1120 ts_tablet_manager.cc:1428] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260927 07:05:17.808480 1125 raft_consensus.cc:482] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:05:17.808779 1125 raft_consensus.cc:504] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } }
I20260927 07:05:17.810477 1125 leader_election.cc:288] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bcf1b0109e749ae97b6be72a898c62e (127.13.121.195:33677), 847a6e6dd7944d8ebd9f673fe12ab746 (127.13.121.194:39447)
I20260927 07:05:17.819278 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.810909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:56087 (local address 127.13.121.195:33677)
0927 07:05:17.811120 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.811126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.811786 (+ 660us) server_negotiation.cc:407] Connection header received
0927 07:05:17.811972 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.811977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.812059 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.812158 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:17.812889 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.813984 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.816810 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.818053 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.818225 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.818304 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.818839 (+ 535us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.818981 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:05:17.819278 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.811005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:38979 (local address 127.13.121.194:39447)
0927 07:05:17.811235 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.811241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.811786 (+ 545us) server_negotiation.cc:407] Connection header received
0927 07:05:17.812004 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.812017 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.812089 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.812168 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:17.812897 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.813984 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.816821 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.818126 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.818262 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.818328 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.818839 (+ 511us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.818985 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:05:17.819463 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.810933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:33677 (local address 127.13.121.193:56087)
0927 07:05:17.811669 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:17.811682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:17.811836 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:17.812186 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:17.812191 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:17.812236 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:17.812691 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.812706 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.814138 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.814142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.816645 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.816654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.818333 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.818341 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.818602 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.818677 (+ 75us) client_negotiation.cc:769] Sending connection context
0927 07:05:17.818778 (+ 101us) client_negotiation.cc:240] Negotiation successful
0927 07:05:17.818963 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":254,"spinlock_wait_cycles":17408,"thread_start_us":379,"threads_started":1}
I20260927 07:05:17.819501 1128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.810772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:39447 (local address 127.13.121.193:38979)
0927 07:05:17.811660 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:17.811681 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:17.811870 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:17.812186 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:17.812191 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:17.812239 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:17.812766 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.812775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.814136 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.814142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.816636 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:17.816654 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.818346 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.818352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:17.818597 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.818674 (+ 77us) client_negotiation.cc:769] Sending connection context
0927 07:05:17.818778 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 07:05:17.818963 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":178,"spinlock_wait_cycles":47232,"thread_start_us":627,"threads_started":1}
I20260927 07:05:17.820780 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b693c1232ae345238efbe716f1061189" candidate_uuid: "4043dbfcf1004ff489a44e528eec6c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" is_pre_election: true
I20260927 07:05:17.820780 1050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b693c1232ae345238efbe716f1061189" candidate_uuid: "4043dbfcf1004ff489a44e528eec6c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bcf1b0109e749ae97b6be72a898c62e" is_pre_election: true
I20260927 07:05:17.821353 916 raft_consensus.cc:2454] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4043dbfcf1004ff489a44e528eec6c9a in term 0.
I20260927 07:05:17.821353 1050 raft_consensus.cc:2454] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4043dbfcf1004ff489a44e528eec6c9a in term 0.
I20260927 07:05:17.822114 711 leader_election.cc:302] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bcf1b0109e749ae97b6be72a898c62e, 4043dbfcf1004ff489a44e528eec6c9a; no voters:
I20260927 07:05:17.822574 1125 raft_consensus.cc:2781] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:05:17.822717 1125 raft_consensus.cc:482] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:05:17.822815 1125 raft_consensus.cc:3037] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:17.825171 1125 raft_consensus.cc:504] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } }
I20260927 07:05:17.825798 1125 leader_election.cc:288] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [CANDIDATE]: Term 1 election: Requested vote from peers 0bcf1b0109e749ae97b6be72a898c62e (127.13.121.195:33677), 847a6e6dd7944d8ebd9f673fe12ab746 (127.13.121.194:39447)
I20260927 07:05:17.826357 1050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b693c1232ae345238efbe716f1061189" candidate_uuid: "4043dbfcf1004ff489a44e528eec6c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bcf1b0109e749ae97b6be72a898c62e"
I20260927 07:05:17.826565 1050 raft_consensus.cc:3037] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:17.828876 1050 raft_consensus.cc:2454] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4043dbfcf1004ff489a44e528eec6c9a in term 1.
I20260927 07:05:17.829231 916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b693c1232ae345238efbe716f1061189" candidate_uuid: "4043dbfcf1004ff489a44e528eec6c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847a6e6dd7944d8ebd9f673fe12ab746"
I20260927 07:05:17.829335 711 leader_election.cc:302] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bcf1b0109e749ae97b6be72a898c62e, 4043dbfcf1004ff489a44e528eec6c9a; no voters:
I20260927 07:05:17.829502 916 raft_consensus.cc:3037] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:17.829653 1125 raft_consensus.cc:2781] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:17.830112 1125 raft_consensus.cc:686] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 1 LEADER]: Becoming Leader. State: Replica: 4043dbfcf1004ff489a44e528eec6c9a, State: Running, Role: LEADER
I20260927 07:05:17.830772 1125 consensus_queue.cc:237] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } }
W20260927 07:05:17.831296 1101 tablet.cc:2369] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:05:17.832494 916 raft_consensus.cc:2454] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4043dbfcf1004ff489a44e528eec6c9a in term 1.
I20260927 07:05:17.835513 630 catalog_manager.cc:5495] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4043dbfcf1004ff489a44e528eec6c9a (127.13.121.193). New cstate: current_term: 1 leader_uuid: "4043dbfcf1004ff489a44e528eec6c9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4043dbfcf1004ff489a44e528eec6c9a" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 36877 } health_report { overall_health: HEALTHY } } }
W20260927 07:05:17.838764 962 tablet.cc:2369] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:05:17.845562 828 tablet.cc:2369] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:05:17.885934 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.880835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37888 (local address 127.13.121.193:36877)
0927 07:05:17.881079 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.881084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.881103 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:05:17.881246 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.881251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.881346 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.881465 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:17.881765 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.882460 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.883844 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.884262 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.884353 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.884893 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:17.885400 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:17.885507 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.885611 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.885680 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260927 07:05:17.894182 916 raft_consensus.cc:1261] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [term 1 FOLLOWER]: Refusing update from remote peer 4043dbfcf1004ff489a44e528eec6c9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:17.894181 1050 raft_consensus.cc:1261] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [term 1 FOLLOWER]: Refusing update from remote peer 4043dbfcf1004ff489a44e528eec6c9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:17.895128 1129 consensus_queue.cc:1035] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "847a6e6dd7944d8ebd9f673fe12ab746" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 39447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:17.895531 1125 consensus_queue.cc:1035] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bcf1b0109e749ae97b6be72a898c62e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 33677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:17.906087 1132 mvcc.cc:205] Tried to move back new op lower bound from 7333858172484886528 to 7333858172239081472. Current Snapshot: MvccSnapshot[applied={T|T < 7333858172484886528}]
I20260927 07:05:17.924039 631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:17.924361 631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:17.928100 631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30a8bcc876d4442a978e4a89bd00f889: Sending DeleteTablet for 3 replicas of tablet b693c1232ae345238efbe716f1061189
I20260927 07:05:17.929450 755 tablet_service.cc:1514] Processing DeleteTablet for tablet b693c1232ae345238efbe716f1061189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:17 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:37872
I20260927 07:05:17.929450 896 tablet_service.cc:1514] Processing DeleteTablet for tablet b693c1232ae345238efbe716f1061189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:17 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:46540
I20260927 07:05:17.929517 1029 tablet_service.cc:1514] Processing DeleteTablet for tablet b693c1232ae345238efbe716f1061189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:17 UTC) from {username='kudu', principal='kudu/127.13.121.254@KRBTEST.COM'} at 127.0.0.1:35104
I20260927 07:05:17.931418 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 692
I20260927 07:05:17.938324 1142 tablet_replica.cc:330] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: stopping tablet replica
I20260927 07:05:17.938979 1142 raft_consensus.cc:2229] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:05:17.939818 1142 raft_consensus.cc:2258] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:17.945240 1143 tablet_replica.cc:330] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: stopping tablet replica
I20260927 07:05:17.945725 1143 raft_consensus.cc:2229] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:17.946281 1143 raft_consensus.cc:2258] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:17.947045 1144 tablet_replica.cc:330] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: stopping tablet replica
I20260927 07:05:17.947471 1144 raft_consensus.cc:2229] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:17.947871 1142 ts_tablet_manager.cc:1905] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:17.947965 1144 raft_consensus.cc:2258] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:17.949951 1144 ts_tablet_manager.cc:1905] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 27 07:05:17 dist-test-slave-p6hz krb5kdc[586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.193@KRBTEST.COM
I20260927 07:05:17.956797 1143 ts_tablet_manager.cc:1905] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:17.956862 1142 ts_tablet_manager.cc:1918] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:17.957036 1142 log.cc:1196] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/b693c1232ae345238efbe716f1061189
I20260927 07:05:17.957605 1142 ts_tablet_manager.cc:1939] T b693c1232ae345238efbe716f1061189 P 4043dbfcf1004ff489a44e528eec6c9a: Deleting consensus metadata
I20260927 07:05:17.959568 616 catalog_manager.cc:4841] TS 4043dbfcf1004ff489a44e528eec6c9a (127.13.121.193:36877): tablet b693c1232ae345238efbe716f1061189 (table test-table [id=04e026f758e94f10b552a1377b7c1b09]) successfully deleted
I20260927 07:05:17.963641 1144 ts_tablet_manager.cc:1918] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:17.963835 1144 log.cc:1196] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/b693c1232ae345238efbe716f1061189
I20260927 07:05:17.964668 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:17.934243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37900 (local address 127.13.121.193:36877)
0927 07:05:17.934429 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:17.934435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:17.944732 (+ 10297us) server_negotiation.cc:407] Connection header received
0927 07:05:17.945226 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:17.945233 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:17.945324 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:17.945401 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:17.947557 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.948625 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.950475 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:17.951259 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:17.951354 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:17.956828 (+ 5474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:17.956857 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:17.956875 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:17.956934 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:17.960862 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:17.961818 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:17.961827 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:17.961834 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:17.962021 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:17.962518 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:17.962524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:17.962527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:17.963017 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:17.963429 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:17.964021 (+ 592us) server_negotiation.cc:299] Negotiation successful
0927 07:05:17.964294 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 07:05:17.964686 1144 ts_tablet_manager.cc:1939] T b693c1232ae345238efbe716f1061189 P 0bcf1b0109e749ae97b6be72a898c62e: Deleting consensus metadata
I20260927 07:05:17.966162 822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:17.967321 618 catalog_manager.cc:4841] TS 0bcf1b0109e749ae97b6be72a898c62e (127.13.121.195:33677): tablet b693c1232ae345238efbe716f1061189 (table test-table [id=04e026f758e94f10b552a1377b7c1b09]) successfully deleted
I20260927 07:05:17.967885 1143 ts_tablet_manager.cc:1918] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:17.968060 1143 log.cc:1196] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/b693c1232ae345238efbe716f1061189
I20260927 07:05:17.968580 1143 ts_tablet_manager.cc:1939] T b693c1232ae345238efbe716f1061189 P 847a6e6dd7944d8ebd9f673fe12ab746: Deleting consensus metadata
I20260927 07:05:17.970395 617 catalog_manager.cc:4841] TS 847a6e6dd7944d8ebd9f673fe12ab746 (127.13.121.194:39447): tablet b693c1232ae345238efbe716f1061189 (table test-table [id=04e026f758e94f10b552a1377b7c1b09]) successfully deleted
I20260927 07:05:18.063268 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 692
I20260927 07:05:18.078548 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 833
Sep 27 07:05:18 dist-test-slave-p6hz krb5kdc[586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.194@KRBTEST.COM
I20260927 07:05:18.095750 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:18.079309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.13.121.194:39447)
0927 07:05:18.079495 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:18.079500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:18.079540 (+ 40us) server_negotiation.cc:407] Connection header received
0927 07:05:18.079704 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:18.079710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:18.079773 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:18.079867 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:18.081136 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:18.081860 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:18.083117 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:18.083560 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:18.083677 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:18.088819 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:18.088835 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:18.088852 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:18.088902 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:18.092726 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:18.093523 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:18.093533 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:18.093536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:18.093698 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:18.094068 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:18.094072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:18.094076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:18.094556 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:18.094844 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:18.095117 (+ 273us) server_negotiation.cc:299] Negotiation successful
0927 07:05:18.095395 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 07:05:18.096882 956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:18.189584 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 833
I20260927 07:05:18.203910 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 966
Sep 27 07:05:18 dist-test-slave-p6hz krb5kdc[586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.195@KRBTEST.COM
I20260927 07:05:18.218842 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:18.204645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35106 (local address 127.13.121.195:33677)
0927 07:05:18.204816 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:18.204821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:18.204839 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:05:18.204978 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:18.204982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:18.205051 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:18.205139 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:18.206428 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:18.207257 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:18.208546 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:18.208961 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:18.209082 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:18.213336 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:18.213352 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:18.213368 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:18.213404 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:18.216200 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:18.216911 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:18.216920 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:18.216922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:18.217128 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:18.217417 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:18.217419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:18.217421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:18.217830 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:18.218077 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:18.218323 (+ 246us) server_negotiation.cc:299] Negotiation successful
0927 07:05:18.218509 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:05:18.219533 1095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:18.307351 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 966
I20260927 07:05:18.321586 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 599
I20260927 07:05:18.322680 661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:18.433164 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 599
2026-09-27T07:05:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4174 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-27T07:05:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:18Z Disabled control of system clock
I20260927 07:05:18.503871 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41501
--webserver_interface=127.13.121.254
--webserver_port=0
--builtin_ntp_servers=127.13.121.212:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:05:18.830552 1153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:18.830861 1153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:18.830942 1153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:18.838261 1153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:18.838387 1153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:18.838424 1153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:18.838483 1153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:18.838726 1153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:18.848272 1153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41501
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:18.849484 1153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:18.851235 1153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:18.862453 1158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:18.862540 1159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:18.862586 1161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:18.863858 1153 server_base.cc:1034] running on GCE node
I20260927 07:05:18.864696 1153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:18.866271 1153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:18.867486 1153 hybrid_clock.cc:648] HybridClock initialized: now 1790492718867430 us; error 42 us; skew 500 ppm
I20260927 07:05:18.867851 1153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:18.870353 1153 webserver.cc:460] Webserver started at http://127.13.121.254:43055/ using document root <none> and password file <none>
I20260927 07:05:18.871070 1153 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:18.871214 1153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:18.871546 1153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:18.874440 1153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "dc618305f22d448f94b651c4121adfe9"
format_stamp: "Formatted at 2026-09-27 07:05:18 on dist-test-slave-p6hz"
server_key: "8ac2265e00f3aba0dd75222e606afe90"
server_key_iv: "133ec270de2911ff573980984de52598"
server_key_version: "encryptionkey@0"
I20260927 07:05:18.875284 1153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "dc618305f22d448f94b651c4121adfe9"
format_stamp: "Formatted at 2026-09-27 07:05:18 on dist-test-slave-p6hz"
server_key: "8ac2265e00f3aba0dd75222e606afe90"
server_key_iv: "133ec270de2911ff573980984de52598"
server_key_version: "encryptionkey@0"
I20260927 07:05:18.880383 1153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:05:18.884060 1167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:18.885510 1153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:18.885715 1153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "dc618305f22d448f94b651c4121adfe9"
format_stamp: "Formatted at 2026-09-27 07:05:18 on dist-test-slave-p6hz"
server_key: "8ac2265e00f3aba0dd75222e606afe90"
server_key_iv: "133ec270de2911ff573980984de52598"
server_key_version: "encryptionkey@0"
I20260927 07:05:18.885941 1153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:18.900705 1153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:18.901513 1153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:18.901754 1153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:18.918505 1218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:41501 every 8 connection(s)
I20260927 07:05:18.918507 1153 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:41501
I20260927 07:05:18.919939 1153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:18.923563 1219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:18.926743 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 1153
I20260927 07:05:18.926920 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:18.927474 13799 external_mini_cluster.cc:1428] Setting key a0e80c742ad9818af75f08044a40d4ba
I20260927 07:05:18.935762 1219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9: Bootstrap starting.
I20260927 07:05:18.939210 1219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:18.940317 1219 log.cc:824] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:18.940726 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:18.929371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.13.121.254:41501)
0927 07:05:18.930307 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:18.930317 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:18.930358 (+ 41us) server_negotiation.cc:407] Connection header received
0927 07:05:18.931993 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:18.932020 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:18.932482 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:18.932813 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:18.933345 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:18.934590 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:18.936353 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:18.937059 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:18.937207 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:18.938311 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:18.938318 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:18.938330 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:18.938982 (+ 652us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:18.939081 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:18.939108 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:18.939267 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:18.939477 (+ 210us) server_negotiation.cc:299] Negotiation successful
0927 07:05:18.939808 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":336,"threads_started":1}
I20260927 07:05:18.943351 1219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9: No bootstrap required, opened a new log
I20260927 07:05:18.950369 1219 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc618305f22d448f94b651c4121adfe9" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41501 } }
I20260927 07:05:18.950726 1219 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:18.950820 1219 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc618305f22d448f94b651c4121adfe9, State: Initialized, Role: FOLLOWER
I20260927 07:05:18.951416 1219 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc618305f22d448f94b651c4121adfe9" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41501 } }
I20260927 07:05:18.951637 1219 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:18.951758 1219 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:18.951906 1219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:18.953962 1219 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc618305f22d448f94b651c4121adfe9" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41501 } }
I20260927 07:05:18.954468 1219 leader_election.cc:302] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc618305f22d448f94b651c4121adfe9; no voters:
I20260927 07:05:18.955065 1219 leader_election.cc:288] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:18.955243 1224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:18.955899 1224 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [term 1 LEADER]: Becoming Leader. State: Replica: dc618305f22d448f94b651c4121adfe9, State: Running, Role: LEADER
I20260927 07:05:18.956405 1224 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc618305f22d448f94b651c4121adfe9" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41501 } }
I20260927 07:05:18.957070 1219 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:18.958859 1225 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc618305f22d448f94b651c4121adfe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc618305f22d448f94b651c4121adfe9" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41501 } } }
I20260927 07:05:18.959295 1225 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:18.959263 1226 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc618305f22d448f94b651c4121adfe9. Latest consensus state: current_term: 1 leader_uuid: "dc618305f22d448f94b651c4121adfe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc618305f22d448f94b651c4121adfe9" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41501 } } }
I20260927 07:05:18.961707 1226 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:18.963945 1232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:18.969134 1232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:18.978529 1232 catalog_manager.cc:1347] Generated new cluster ID: 860f31dbf3be4f42b2014b228ee0554d
I20260927 07:05:18.978645 1232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:18.994609 1232 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:18.995918 1232 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:19.011655 1232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9: Generated new TSK 0
I20260927 07:05:19.012792 1232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:19.029798 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.193:0
--local_ip_for_outbound_sockets=127.13.121.193
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41501
--builtin_ntp_servers=127.13.121.212:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 07:05:19.369865 1243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:19.370157 1243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:19.370246 1243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:19.377574 1243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:19.377684 1243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:19.377791 1243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.193
I20260927 07:05:19.387013 1243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.121.193
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.193
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:19.388271 1243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:19.390033 1243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:19.403038 1248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:19.403076 1249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:19.403008 1251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:19.403508 1243 server_base.cc:1034] running on GCE node
I20260927 07:05:19.404213 1243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:19.405372 1243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:19.406620 1243 hybrid_clock.cc:648] HybridClock initialized: now 1790492719406559 us; error 46 us; skew 500 ppm
I20260927 07:05:19.406971 1243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:19.409307 1243 webserver.cc:460] Webserver started at http://127.13.121.193:36667/ using document root <none> and password file <none>
I20260927 07:05:19.409983 1243 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:19.410075 1243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:19.410378 1243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:19.413273 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/data/instance:
uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb"
format_stamp: "Formatted at 2026-09-27 07:05:19 on dist-test-slave-p6hz"
server_key: "877e7f745ad93ab6a4e5758f25d4c8c2"
server_key_iv: "991afbf45cd165a3a5f245d1385cc4b4"
server_key_version: "encryptionkey@0"
I20260927 07:05:19.414031 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance:
uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb"
format_stamp: "Formatted at 2026-09-27 07:05:19 on dist-test-slave-p6hz"
server_key: "877e7f745ad93ab6a4e5758f25d4c8c2"
server_key_iv: "991afbf45cd165a3a5f245d1385cc4b4"
server_key_version: "encryptionkey@0"
I20260927 07:05:19.419282 1243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260927 07:05:19.422878 1257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:19.424449 1243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:19.424626 1243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/wal
uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb"
format_stamp: "Formatted at 2026-09-27 07:05:19 on dist-test-slave-p6hz"
server_key: "877e7f745ad93ab6a4e5758f25d4c8c2"
server_key_iv: "991afbf45cd165a3a5f245d1385cc4b4"
server_key_version: "encryptionkey@0"
I20260927 07:05:19.424853 1243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:19.464481 1243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:19.465314 1243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:19.465586 1243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:19.466528 1243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:19.468024 1243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:19.468113 1243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:19.468228 1243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:19.468304 1243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:19.497375 1243 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.193:45215
I20260927 07:05:19.497421 1369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.193:45215 every 8 connection(s)
I20260927 07:05:19.498600 1243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/data/info.pb
I20260927 07:05:19.501011 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 1243
I20260927 07:05:19.501206 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/wal/instance
I20260927 07:05:19.501670 13799 external_mini_cluster.cc:1428] Setting key ad54555e70f3109c8ecf5fa50ffee2e8
I20260927 07:05:19.504505 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.194:0
--local_ip_for_outbound_sockets=127.13.121.194
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41501
--builtin_ntp_servers=127.13.121.212:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:05:19.513357 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:19.501040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.193:43109 (local address 127.13.121.254:41501)
0927 07:05:19.501523 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:19.501529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:19.502732 (+ 1203us) server_negotiation.cc:407] Connection header received
0927 07:05:19.505165 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:19.505172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:19.505262 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:19.505366 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:19.506812 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:19.507944 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:19.510284 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:19.510917 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:19.511044 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:19.512518 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:19.512523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:19.512526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:19.512546 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:19.512584 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:19.512597 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:19.512670 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:19.512900 (+ 230us) server_negotiation.cc:299] Negotiation successful
0927 07:05:19.513002 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":234,"threads_started":1}
I20260927 07:05:19.514379 1373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:19.501585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41501 (local address 127.13.121.193:43109)
0927 07:05:19.502564 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:19.502618 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:19.504846 (+ 2228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:19.505714 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:19.505726 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:19.505855 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:19.506584 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:19.506614 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:19.508165 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:19.508171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:19.510080 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:19.510095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:19.511330 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:19.511337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:19.511543 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:19.512201 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:05:19.512203 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:19.512243 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:05:19.512329 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:19.512696 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:19.512707 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:19.512715 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 07:05:19.512875 (+ 160us) client_negotiation.cc:240] Negotiation successful
0927 07:05:19.513288 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":296,"threads_started":1}
I20260927 07:05:19.515902 1370 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41501
I20260927 07:05:19.516253 1370 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:19.517131 1370 heartbeater.cc:511] Master 127.13.121.254:41501 requested a full tablet report, sending...
I20260927 07:05:19.519408 1184 ts_manager.cc:194] Registered new tserver with Master: cc1ed24cf7eb43adb3d8fe113a69c8eb (127.13.121.193:45215)
I20260927 07:05:19.520854 1184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.193:43109
W20260927 07:05:19.830940 1374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:19.831293 1374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:19.831426 1374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:19.838937 1374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:19.839051 1374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:19.839211 1374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.194
I20260927 07:05:19.848940 1374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.121.194
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.194
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:19.850221 1374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:19.852020 1374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:19.864831 1379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:19.864866 1382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:19.864888 1380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:19.866008 1374 server_base.cc:1034] running on GCE node
I20260927 07:05:19.866636 1374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:19.867718 1374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:19.868930 1374 hybrid_clock.cc:648] HybridClock initialized: now 1790492719868867 us; error 53 us; skew 500 ppm
I20260927 07:05:19.869263 1374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:19.871603 1374 webserver.cc:460] Webserver started at http://127.13.121.194:45603/ using document root <none> and password file <none>
I20260927 07:05:19.872314 1374 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:19.872406 1374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:19.872728 1374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:19.875679 1374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/data/instance:
uuid: "e356f545fcfe437e84d93f706f517412"
format_stamp: "Formatted at 2026-09-27 07:05:19 on dist-test-slave-p6hz"
server_key: "9bea153e2e9393b79164f912c7cdbc1b"
server_key_iv: "fab76bd31cd69f2235b4089f8af08801"
server_key_version: "encryptionkey@0"
I20260927 07:05:19.876456 1374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance:
uuid: "e356f545fcfe437e84d93f706f517412"
format_stamp: "Formatted at 2026-09-27 07:05:19 on dist-test-slave-p6hz"
server_key: "9bea153e2e9393b79164f912c7cdbc1b"
server_key_iv: "fab76bd31cd69f2235b4089f8af08801"
server_key_version: "encryptionkey@0"
I20260927 07:05:19.881474 1374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:05:19.884999 1388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:19.886298 1374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:05:19.886442 1374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/wal
uuid: "e356f545fcfe437e84d93f706f517412"
format_stamp: "Formatted at 2026-09-27 07:05:19 on dist-test-slave-p6hz"
server_key: "9bea153e2e9393b79164f912c7cdbc1b"
server_key_iv: "fab76bd31cd69f2235b4089f8af08801"
server_key_version: "encryptionkey@0"
I20260927 07:05:19.886643 1374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:19.903043 1374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:19.903939 1374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:19.904229 1374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:19.905200 1374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:19.906657 1374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:19.906745 1374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:19.906852 1374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:19.906915 1374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:19.939478 1374 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.194:37879
I20260927 07:05:19.939514 1500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.194:37879 every 8 connection(s)
I20260927 07:05:19.940829 1374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/data/info.pb
I20260927 07:05:19.943048 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 1374
I20260927 07:05:19.943284 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/wal/instance
I20260927 07:05:19.943728 13799 external_mini_cluster.cc:1428] Setting key b1c03f1404b9b99dbb4ed338ede79631
I20260927 07:05:19.946980 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.121.195:0
--local_ip_for_outbound_sockets=127.13.121.195
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41501
--builtin_ntp_servers=127.13.121.212:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 07:05:19.953760 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:19.943545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:37399 (local address 127.13.121.254:41501)
0927 07:05:19.943777 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:19.943784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:19.945286 (+ 1502us) server_negotiation.cc:407] Connection header received
0927 07:05:19.947345 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:19.947351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:19.947426 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:19.947512 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:19.948741 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:19.949440 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:19.950998 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:19.951453 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:19.951576 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:19.952935 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:19.952940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:19.952943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:19.952963 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:19.952996 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:19.953009 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:19.953081 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:19.953346 (+ 265us) server_negotiation.cc:299] Negotiation successful
0927 07:05:19.953452 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 07:05:19.954811 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:19.943984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41501 (local address 127.13.121.194:37399)
0927 07:05:19.945006 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:19.945054 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:19.946941 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:19.947841 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:19.947851 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:19.947955 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:19.948567 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:19.948585 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:19.949591 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:19.949595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:19.950844 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:19.950857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:19.951676 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:19.951684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:19.951916 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:19.952620 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:05:19.952622 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:19.952664 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:05:19.952746 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:19.953134 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:19.953143 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:19.953149 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 07:05:19.953307 (+ 158us) client_negotiation.cc:240] Negotiation successful
0927 07:05:19.953729 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":284,"threads_started":1}
I20260927 07:05:19.956323 1501 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41501
I20260927 07:05:19.956624 1501 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:19.957396 1501 heartbeater.cc:511] Master 127.13.121.254:41501 requested a full tablet report, sending...
I20260927 07:05:19.958916 1184 ts_manager.cc:194] Registered new tserver with Master: e356f545fcfe437e84d93f706f517412 (127.13.121.194:37879)
I20260927 07:05:19.959746 1184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.194:37399
W20260927 07:05:20.274077 1504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:20.274379 1504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:20.274606 1504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:20.282099 1504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:20.282224 1504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:20.282371 1504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.121.195
I20260927 07:05:20.291952 1504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.121.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.121.195
--webserver_port=0
--tserver_master_addrs=127.13.121.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.121.195
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:20.293291 1504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:20.295092 1504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:20.308856 1512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:20.308861 1509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:20.308897 1510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:20.309813 1504 server_base.cc:1034] running on GCE node
I20260927 07:05:20.310418 1504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:20.311429 1504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:20.312662 1504 hybrid_clock.cc:648] HybridClock initialized: now 1790492720312608 us; error 44 us; skew 500 ppm
I20260927 07:05:20.312964 1504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:05:20.315207 1504 webserver.cc:460] Webserver started at http://127.13.121.195:39819/ using document root <none> and password file <none>
I20260927 07:05:20.315909 1504 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:20.316008 1504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:20.316303 1504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:20.319087 1504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/data/instance:
uuid: "045ec8bc4bd04b54be0eada5f693396e"
format_stamp: "Formatted at 2026-09-27 07:05:20 on dist-test-slave-p6hz"
server_key: "66b2f93e7c82221ca3e98c59ebb36473"
server_key_iv: "3b36ef483021afef6a3c6a696b2eb7b0"
server_key_version: "encryptionkey@0"
I20260927 07:05:20.319929 1504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance:
uuid: "045ec8bc4bd04b54be0eada5f693396e"
format_stamp: "Formatted at 2026-09-27 07:05:20 on dist-test-slave-p6hz"
server_key: "66b2f93e7c82221ca3e98c59ebb36473"
server_key_iv: "3b36ef483021afef6a3c6a696b2eb7b0"
server_key_version: "encryptionkey@0"
I20260927 07:05:20.325145 1504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260927 07:05:20.328784 1518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:20.330209 1504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:05:20.330375 1504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/wal
uuid: "045ec8bc4bd04b54be0eada5f693396e"
format_stamp: "Formatted at 2026-09-27 07:05:20 on dist-test-slave-p6hz"
server_key: "66b2f93e7c82221ca3e98c59ebb36473"
server_key_iv: "3b36ef483021afef6a3c6a696b2eb7b0"
server_key_version: "encryptionkey@0"
I20260927 07:05:20.330571 1504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:20.354043 1504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:20.354908 1504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:20.355237 1504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:20.356289 1504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 07:05:20.357827 1504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 07:05:20.357913 1504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:20.358008 1504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 07:05:20.358080 1504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:20.388348 1504 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.195:37539
I20260927 07:05:20.388417 1630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.195:37539 every 8 connection(s)
I20260927 07:05:20.389647 1504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/data/info.pb
I20260927 07:05:20.391510 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 1504
I20260927 07:05:20.391708 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/wal/instance
I20260927 07:05:20.392132 13799 external_mini_cluster.cc:1428] Setting key 4c98d31456a8083689c3a673c1994e59
I20260927 07:05:20.402199 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.392341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.195:47257 (local address 127.13.121.254:41501)
0927 07:05:20.392558 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.392563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.394202 (+ 1639us) server_negotiation.cc:407] Connection header received
0927 07:05:20.396148 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.396153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.396226 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.396308 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:20.397309 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.398031 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.399724 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.400113 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.400222 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.401513 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:20.401518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:20.401521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:20.401548 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:20.401582 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:20.401596 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:20.401670 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.401853 (+ 183us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.401941 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 07:05:20.403194 1633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.392909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41501 (local address 127.13.121.195:47257)
0927 07:05:20.394074 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:20.394116 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:20.395906 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:20.396483 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:20.396492 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:20.396590 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:20.397181 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.397199 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.398166 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.398170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.399612 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.399624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.400324 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.400329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.400532 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.401158 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 07:05:20.401160 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:20.401210 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 07:05:20.401294 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:20.401670 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:20.401676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:20.401681 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 07:05:20.401826 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 07:05:20.402155 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":374,"threads_started":1}
I20260927 07:05:20.404563 1631 heartbeater.cc:347] Connected to a master server at 127.13.121.254:41501
I20260927 07:05:20.404873 1631 heartbeater.cc:464] Registering TS with master...
I20260927 07:05:20.405815 1631 heartbeater.cc:511] Master 127.13.121.254:41501 requested a full tablet report, sending...
I20260927 07:05:20.407464 1184 ts_manager.cc:194] Registered new tserver with Master: 045ec8bc4bd04b54be0eada5f693396e (127.13.121.195:37539)
I20260927 07:05:20.408291 1184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.121.195:47257
I20260927 07:05:20.420110 13799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 07:05:20.429930 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.424145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.13.121.254:41501)
0927 07:05:20.424312 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.424317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.424684 (+ 367us) server_negotiation.cc:407] Connection header received
0927 07:05:20.424822 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.424826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.424898 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.424982 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:20.425367 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.426071 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.427571 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.428062 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.428217 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.429277 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:20.429283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:20.429286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:20.429311 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:20.429350 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:20.429362 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:20.429431 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.429521 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.429610 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 07:05:20.433700 1184 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 07:05:20.440681 1184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 07:05:20.474980 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.453459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58282 (local address 127.13.121.193:45215)
0927 07:05:20.464668 (+ 11209us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.464679 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.466066 (+ 1387us) server_negotiation.cc:407] Connection header received
0927 07:05:20.466158 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.466164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.466386 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.467347 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:20.467608 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.469054 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.472383 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.473841 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.473995 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.474211 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.474490 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.474662 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10964,"thread_start_us":10807,"threads_started":1}
I20260927 07:05:20.475031 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.453214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45215 (local address 127.0.0.1:58282)
0927 07:05:20.465707 (+ 12493us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:20.465723 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:20.465842 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:20.466621 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:20.466628 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:20.466659 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:20.467101 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.467115 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.469186 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.469191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.472101 (+ 2910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.472113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.474021 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.474027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.474227 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.474287 (+ 60us) client_negotiation.cc:769] Sending connection context
0927 07:05:20.474396 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 07:05:20.474583 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12370,"mutex_wait_us":217,"spinlock_wait_cycles":17152,"thread_start_us":12219,"threads_started":1}
I20260927 07:05:20.475206 1643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.453049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.194:37879 (local address 127.0.0.1:41316)
0927 07:05:20.465529 (+ 12480us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:20.465598 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:20.465843 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:20.466807 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:20.466811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:20.466830 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:20.467243 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.467254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.469176 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.469182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.472080 (+ 2898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.472096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.474352 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.474358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.474516 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.474556 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 07:05:20.474636 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 07:05:20.474762 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12173,"mutex_wait_us":233,"thread_start_us":11697,"threads_started":1}
I20260927 07:05:20.476050 1645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.453307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.13.121.194:37879)
0927 07:05:20.465396 (+ 12089us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.465405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.465738 (+ 333us) server_negotiation.cc:407] Connection header received
0927 07:05:20.466125 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.466133 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.466389 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.466586 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:20.467440 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.469009 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.472442 (+ 3433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.473842 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.474008 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.474230 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.475568 (+ 1338us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.475734 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11899,"thread_start_us":11784,"threads_started":1}
I20260927 07:05:20.479589 1648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.467201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:37539 (local address 127.0.0.1:58626)
0927 07:05:20.471733 (+ 4532us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:20.471752 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:20.471877 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:20.472514 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:20.472520 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:20.472547 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:20.472876 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.472888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.474769 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.474773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.477384 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.477398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.478838 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.478845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.479010 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.479041 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 07:05:20.479151 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 07:05:20.479279 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4386,"thread_start_us":3130,"threads_started":1}
I20260927 07:05:20.479642 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.466445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58626 (local address 127.13.121.195:37539)
0927 07:05:20.470934 (+ 4489us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.470940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.471859 (+ 919us) server_negotiation.cc:407] Connection header received
0927 07:05:20.472028 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.472036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.472322 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.472526 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:20.473109 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.474623 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.477600 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.478649 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.478832 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.479052 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.479193 (+ 141us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.479347 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4318,"thread_start_us":4161,"threads_started":1}
I20260927 07:05:20.479238 1436 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6f697dfa864d4ca69a9d36753d3557 (DEFAULT_TABLE table=test-table [id=86123f7b74404960b616f929ac093538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:20.479238 1305 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6f697dfa864d4ca69a9d36753d3557 (DEFAULT_TABLE table=test-table [id=86123f7b74404960b616f929ac093538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:20.480638 1305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6f697dfa864d4ca69a9d36753d3557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:20.480640 1436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6f697dfa864d4ca69a9d36753d3557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:20.482735 1566 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6f697dfa864d4ca69a9d36753d3557 (DEFAULT_TABLE table=test-table [id=86123f7b74404960b616f929ac093538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 07:05:20.484007 1566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6f697dfa864d4ca69a9d36753d3557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:20.494799 1649 tablet_bootstrap.cc:492] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: Bootstrap starting.
I20260927 07:05:20.499457 1649 tablet_bootstrap.cc:654] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:20.500710 1649 log.cc:824] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:20.501016 1650 tablet_bootstrap.cc:492] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: Bootstrap starting.
I20260927 07:05:20.503720 1651 tablet_bootstrap.cc:492] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: Bootstrap starting.
I20260927 07:05:20.506832 1651 tablet_bootstrap.cc:654] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:20.508196 1651 log.cc:824] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:20.509641 1650 tablet_bootstrap.cc:654] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:20.510947 1650 log.cc:824] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:20.513082 1651 tablet_bootstrap.cc:492] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: No bootstrap required, opened a new log
I20260927 07:05:20.513331 1651 ts_tablet_manager.cc:1397] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260927 07:05:20.513695 1650 tablet_bootstrap.cc:492] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: No bootstrap required, opened a new log
I20260927 07:05:20.513706 1649 tablet_bootstrap.cc:492] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: No bootstrap required, opened a new log
I20260927 07:05:20.513962 1650 ts_tablet_manager.cc:1397] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260927 07:05:20.513962 1649 ts_tablet_manager.cc:1397] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.005s
I20260927 07:05:20.520638 1651 raft_consensus.cc:348] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } }
I20260927 07:05:20.521011 1651 raft_consensus.cc:374] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:20.521103 1651 raft_consensus.cc:729] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1ed24cf7eb43adb3d8fe113a69c8eb, State: Initialized, Role: FOLLOWER
I20260927 07:05:20.521741 1651 consensus_queue.cc:260] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } }
I20260927 07:05:20.522925 1370 heartbeater.cc:503] Master 127.13.121.254:41501 was elected leader, sending a full tablet report...
I20260927 07:05:20.523705 1651 ts_tablet_manager.cc:1428] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: Time spent starting tablet: real 0.010s user 0.005s sys 0.007s
I20260927 07:05:20.524302 1650 raft_consensus.cc:348] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } }
I20260927 07:05:20.524302 1649 raft_consensus.cc:348] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } }
I20260927 07:05:20.524755 1649 raft_consensus.cc:374] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:20.524757 1650 raft_consensus.cc:374] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:20.524860 1649 raft_consensus.cc:729] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 045ec8bc4bd04b54be0eada5f693396e, State: Initialized, Role: FOLLOWER
I20260927 07:05:20.524860 1650 raft_consensus.cc:729] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e356f545fcfe437e84d93f706f517412, State: Initialized, Role: FOLLOWER
I20260927 07:05:20.525506 1649 consensus_queue.cc:260] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } }
I20260927 07:05:20.525506 1650 consensus_queue.cc:260] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } }
I20260927 07:05:20.527205 1501 heartbeater.cc:503] Master 127.13.121.254:41501 was elected leader, sending a full tablet report...
I20260927 07:05:20.527369 1631 heartbeater.cc:503] Master 127.13.121.254:41501 was elected leader, sending a full tablet report...
I20260927 07:05:20.527488 1649 ts_tablet_manager.cc:1428] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260927 07:05:20.528504 1650 ts_tablet_manager.cc:1428] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260927 07:05:20.546980 1656 raft_consensus.cc:482] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 07:05:20.547353 1656 raft_consensus.cc:504] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } }
I20260927 07:05:20.548980 1656 leader_election.cc:288] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1ed24cf7eb43adb3d8fe113a69c8eb (127.13.121.193:45215), 045ec8bc4bd04b54be0eada5f693396e (127.13.121.195:37539)
I20260927 07:05:20.555821 1658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.549423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.193:45215 (local address 127.13.121.194:50461)
0927 07:05:20.549998 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:20.550014 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:20.550178 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:20.550587 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:20.550592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:20.550625 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:20.551018 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.551031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.552196 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.552202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.553808 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.553820 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.555091 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.555098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.555287 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.555323 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 07:05:20.555413 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 07:05:20.555552 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":289,"threads_started":1}
I20260927 07:05:20.555866 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.549438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:50461 (local address 127.13.121.193:45215)
0927 07:05:20.549606 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.549610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.550110 (+ 500us) server_negotiation.cc:407] Connection header received
0927 07:05:20.550340 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.550345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.550438 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.550539 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:20.551181 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.552060 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.554005 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.554907 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.555023 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.555073 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.555447 (+ 374us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.555585 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:05:20.556113 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.550117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.195:37539 (local address 127.13.121.194:55463)
0927 07:05:20.550708 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:20.550721 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:20.550821 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:20.551197 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:20.551201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:20.551225 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 07:05:20.551562 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.551572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.552765 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.552769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.554174 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:20.554188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.555421 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.555428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:20.555599 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.555631 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 07:05:20.555720 (+ 89us) client_negotiation.cc:240] Negotiation successful
0927 07:05:20.555846 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":444,"threads_started":1}
I20260927 07:05:20.556160 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.550398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.121.194:55463 (local address 127.13.121.195:37539)
0927 07:05:20.550568 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.550574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.550802 (+ 228us) server_negotiation.cc:407] Connection header received
0927 07:05:20.550935 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.550940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.551026 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.551121 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 07:05:20.551747 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.552631 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.554364 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.555256 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.555409 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.555473 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.555756 (+ 283us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.555880 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 07:05:20.557197 1586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6f697dfa864d4ca69a9d36753d3557" candidate_uuid: "e356f545fcfe437e84d93f706f517412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "045ec8bc4bd04b54be0eada5f693396e" is_pre_election: true
I20260927 07:05:20.557195 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6f697dfa864d4ca69a9d36753d3557" candidate_uuid: "e356f545fcfe437e84d93f706f517412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" is_pre_election: true
I20260927 07:05:20.557709 1325 raft_consensus.cc:2454] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e356f545fcfe437e84d93f706f517412 in term 0.
I20260927 07:05:20.557713 1586 raft_consensus.cc:2454] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e356f545fcfe437e84d93f706f517412 in term 0.
I20260927 07:05:20.558296 1392 leader_election.cc:302] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1ed24cf7eb43adb3d8fe113a69c8eb, e356f545fcfe437e84d93f706f517412; no voters:
I20260927 07:05:20.558660 1656 raft_consensus.cc:2781] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 07:05:20.558785 1656 raft_consensus.cc:482] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 07:05:20.558883 1656 raft_consensus.cc:3037] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:20.561100 1656 raft_consensus.cc:504] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } }
I20260927 07:05:20.561654 1656 leader_election.cc:288] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [CANDIDATE]: Term 1 election: Requested vote from peers cc1ed24cf7eb43adb3d8fe113a69c8eb (127.13.121.193:45215), 045ec8bc4bd04b54be0eada5f693396e (127.13.121.195:37539)
I20260927 07:05:20.562175 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6f697dfa864d4ca69a9d36753d3557" candidate_uuid: "e356f545fcfe437e84d93f706f517412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb"
I20260927 07:05:20.562234 1586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6f697dfa864d4ca69a9d36753d3557" candidate_uuid: "e356f545fcfe437e84d93f706f517412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "045ec8bc4bd04b54be0eada5f693396e"
I20260927 07:05:20.562362 1325 raft_consensus.cc:3037] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:20.562417 1586 raft_consensus.cc:3037] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:20.564695 1586 raft_consensus.cc:2454] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e356f545fcfe437e84d93f706f517412 in term 1.
I20260927 07:05:20.564759 1325 raft_consensus.cc:2454] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e356f545fcfe437e84d93f706f517412 in term 1.
I20260927 07:05:20.565148 1392 leader_election.cc:302] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 045ec8bc4bd04b54be0eada5f693396e, e356f545fcfe437e84d93f706f517412; no voters:
I20260927 07:05:20.565531 1656 raft_consensus.cc:2781] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:20.566037 1656 raft_consensus.cc:686] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 1 LEADER]: Becoming Leader. State: Replica: e356f545fcfe437e84d93f706f517412, State: Running, Role: LEADER
I20260927 07:05:20.566780 1656 consensus_queue.cc:237] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } }
I20260927 07:05:20.571576 1182 catalog_manager.cc:5495] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 reported cstate change: term changed from 0 to 1, leader changed from <none> to e356f545fcfe437e84d93f706f517412 (127.13.121.194). New cstate: current_term: 1 leader_uuid: "e356f545fcfe437e84d93f706f517412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e356f545fcfe437e84d93f706f517412" member_type: VOTER last_known_addr { host: "127.13.121.194" port: 37879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 } health_report { overall_health: UNKNOWN } } }
I20260927 07:05:20.601836 1645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.596644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.13.121.194:37879)
0927 07:05:20.596832 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.596837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.596852 (+ 15us) server_negotiation.cc:407] Connection header received
0927 07:05:20.596936 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.596940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.597036 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.597173 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:20.597567 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.598241 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.599700 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.600129 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.600239 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.600756 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:20.601287 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:20.601400 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.601495 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.601568 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 07:05:20.610239 1586 raft_consensus.cc:1261] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [term 1 FOLLOWER]: Refusing update from remote peer e356f545fcfe437e84d93f706f517412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:20.610237 1325 raft_consensus.cc:1261] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [term 1 FOLLOWER]: Refusing update from remote peer e356f545fcfe437e84d93f706f517412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:20.611277 1660 consensus_queue.cc:1035] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045ec8bc4bd04b54be0eada5f693396e" member_type: VOTER last_known_addr { host: "127.13.121.195" port: 37539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:20.611644 1656 consensus_queue.cc:1035] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1ed24cf7eb43adb3d8fe113a69c8eb" member_type: VOTER last_known_addr { host: "127.13.121.193" port: 45215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:20.635890 1181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:20.636238 1181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 07:05:20.640259 1181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc618305f22d448f94b651c4121adfe9: Sending DeleteTablet for 3 replicas of tablet 4f6f697dfa864d4ca69a9d36753d3557
I20260927 07:05:20.641858 1436 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f6f697dfa864d4ca69a9d36753d3557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:20 UTC) from {username='slave'} at 127.0.0.1:41316
I20260927 07:05:20.642197 1305 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f6f697dfa864d4ca69a9d36753d3557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:20 UTC) from {username='slave'} at 127.0.0.1:58282
I20260927 07:05:20.642827 1566 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f6f697dfa864d4ca69a9d36753d3557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 07:05:20 UTC) from {username='slave'} at 127.0.0.1:58626
I20260927 07:05:20.644201 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 1243
I20260927 07:05:20.655194 1674 tablet_replica.cc:330] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: stopping tablet replica
I20260927 07:05:20.655704 1674 raft_consensus.cc:2229] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:20.656286 1674 raft_consensus.cc:2258] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:20.658780 1673 tablet_replica.cc:330] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: stopping tablet replica
I20260927 07:05:20.659250 1673 raft_consensus.cc:2229] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 07:05:20.659332 1674 ts_tablet_manager.cc:1905] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:20.659775 1673 raft_consensus.cc:2258] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:20.662477 1673 ts_tablet_manager.cc:1905] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:20.667436 1674 ts_tablet_manager.cc:1918] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:20.667708 1674 log.cc:1196] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-2/wal/wals/4f6f697dfa864d4ca69a9d36753d3557
I20260927 07:05:20.668324 1674 ts_tablet_manager.cc:1939] T 4f6f697dfa864d4ca69a9d36753d3557 P 045ec8bc4bd04b54be0eada5f693396e: Deleting consensus metadata
I20260927 07:05:20.669900 1673 ts_tablet_manager.cc:1918] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:20.670074 1673 log.cc:1196] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-0/wal/wals/4f6f697dfa864d4ca69a9d36753d3557
I20260927 07:05:20.670578 1171 catalog_manager.cc:4841] TS 045ec8bc4bd04b54be0eada5f693396e (127.13.121.195:37539): tablet 4f6f697dfa864d4ca69a9d36753d3557 (table test-table [id=86123f7b74404960b616f929ac093538]) successfully deleted
I20260927 07:05:20.671205 1673 ts_tablet_manager.cc:1939] T 4f6f697dfa864d4ca69a9d36753d3557 P cc1ed24cf7eb43adb3d8fe113a69c8eb: Deleting consensus metadata
I20260927 07:05:20.671742 1672 tablet_replica.cc:330] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: stopping tablet replica
I20260927 07:05:20.672307 1672 raft_consensus.cc:2229] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 1 LEADER]: Raft consensus shutting down.
I20260927 07:05:20.673092 1171 catalog_manager.cc:4841] TS cc1ed24cf7eb43adb3d8fe113a69c8eb (127.13.121.193:45215): tablet 4f6f697dfa864d4ca69a9d36753d3557 (table test-table [id=86123f7b74404960b616f929ac093538]) successfully deleted
I20260927 07:05:20.673198 1672 raft_consensus.cc:2258] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 07:05:20.675418 1672 ts_tablet_manager.cc:1905] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 07:05:20.677240 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.645814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.13.121.193:45215)
0927 07:05:20.646014 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.646021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.669982 (+ 23961us) server_negotiation.cc:407] Connection header received
0927 07:05:20.670497 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.670503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.670618 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.670717 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:20.671480 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.672576 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.674249 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.674668 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.674750 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.675900 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:20.675909 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:20.675921 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:20.676465 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:20.676516 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:20.676536 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:20.676631 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.676785 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.676896 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:05:20.677908 1365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:20.683251 1672 ts_tablet_manager.cc:1918] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 07:05:20.683439 1672 log.cc:1196] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: Deleting WAL directory at /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790492406663968-13799-0/minicluster-data/ts-1/wal/wals/4f6f697dfa864d4ca69a9d36753d3557
I20260927 07:05:20.684044 1672 ts_tablet_manager.cc:1939] T 4f6f697dfa864d4ca69a9d36753d3557 P e356f545fcfe437e84d93f706f517412: Deleting consensus metadata
I20260927 07:05:20.685966 1168 catalog_manager.cc:4841] TS e356f545fcfe437e84d93f706f517412 (127.13.121.194:37879): tablet 4f6f697dfa864d4ca69a9d36753d3557 (table test-table [id=86123f7b74404960b616f929ac093538]) successfully deleted
I20260927 07:05:20.766184 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 1243
I20260927 07:05:20.781350 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 1374
I20260927 07:05:20.788472 1645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.782154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41346 (local address 127.13.121.194:37879)
0927 07:05:20.782348 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.782353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.782375 (+ 22us) server_negotiation.cc:407] Connection header received
0927 07:05:20.782516 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.782520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.782614 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.782708 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:20.783118 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.783997 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.785256 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.785730 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.785845 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.786903 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:20.786912 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:20.786928 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:20.787706 (+ 778us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:20.787760 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:20.787785 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:20.787879 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.788017 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.788126 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 07:05:20.789189 1496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:20.900395 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 1374
I20260927 07:05:20.915663 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 1504
I20260927 07:05:20.926088 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:20.916402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58630 (local address 127.13.121.195:37539)
0927 07:05:20.916603 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:20.916607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:20.916625 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:05:20.916720 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:20.916724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:20.916802 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:20.916881 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:20.917215 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.917922 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.919742 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:20.920128 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:20.920240 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:20.924716 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:20.924726 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:20.924741 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:20.925295 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:20.925347 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:20.925368 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:20.925458 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:20.925643 (+ 185us) server_negotiation.cc:299] Negotiation successful
0927 07:05:20.925781 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:05:20.926832 1626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:21.015324 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 1504
I20260927 07:05:21.029163 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 1153
I20260927 07:05:21.030222 1214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:21.138343 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 1153
2026-09-27T07:05:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2708 ms)
[----------] 6 tests from EncryptionPolicyTest (21042 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:21 dist-test-slave-p6hz krb5kdc[1682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:21 dist-test-slave-p6hz krb5kdc[1682](info): set up 2 sockets
Sep 27 07:05:21 dist-test-slave-p6hz krb5kdc[1682](info): commencing operation
krb5kdc: starting...
W20260927 07:05:22.175612 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:22 dist-test-slave-p6hz krb5kdc[1682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:22.419359 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46415
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:22.748888 1695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:22.749212 1695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:22.749348 1695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:22.756549 1695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:22.756681 1695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:22.756763 1695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:22.756850 1695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:22.756911 1695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:22.767128 1695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:22.768558 1695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:22.769905 1695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:22.781395 1703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:22.781450 1700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:22.781430 1701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:22.782291 1695 server_base.cc:1034] running on GCE node
I20260927 07:05:22.783088 1695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:22.784577 1695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:22.785806 1695 hybrid_clock.cc:648] HybridClock initialized: now 1790492722785750 us; error 45 us; skew 500 ppm
I20260927 07:05:22.786194 1695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:22 dist-test-slave-p6hz krb5kdc[1682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492722, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:22.789407 1695 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:22.790902 1695 webserver.cc:460] Webserver started at http://127.0.0.1:33057/ using document root <none> and password file <none>
I20260927 07:05:22.791621 1695 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:22.791746 1695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:22.792032 1695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:22.795050 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "eab7a83c0db64b57bc3da65f72521c87"
format_stamp: "Formatted at 2026-09-27 07:05:22 on dist-test-slave-p6hz"
server_key: "890e98fa9d57a85f0bffaad3a9e07678"
server_key_iv: "5a63b59985824259781f10a5096674fc"
server_key_version: "encryptionkey@0"
I20260927 07:05:22.795959 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "eab7a83c0db64b57bc3da65f72521c87"
format_stamp: "Formatted at 2026-09-27 07:05:22 on dist-test-slave-p6hz"
server_key: "890e98fa9d57a85f0bffaad3a9e07678"
server_key_iv: "5a63b59985824259781f10a5096674fc"
server_key_version: "encryptionkey@0"
I20260927 07:05:22.801433 1695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:05:22.805109 1710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:22.806509 1695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:05:22.806684 1695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "eab7a83c0db64b57bc3da65f72521c87"
format_stamp: "Formatted at 2026-09-27 07:05:22 on dist-test-slave-p6hz"
server_key: "890e98fa9d57a85f0bffaad3a9e07678"
server_key_iv: "5a63b59985824259781f10a5096674fc"
server_key_version: "encryptionkey@0"
I20260927 07:05:22.806875 1695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:22.849364 1695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:22.853698 1695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:22.853942 1695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:22.871436 1695 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46415
I20260927 07:05:22.871459 1761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46415 every 8 connection(s)
I20260927 07:05:22.872891 1695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:22.876590 1762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:22.883215 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 1695
I20260927 07:05:22.883415 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:22.884006 13799 external_mini_cluster.cc:1428] Setting key a324b2d0b77d827521d580f983ca5c52
I20260927 07:05:22.887462 1762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87: Bootstrap starting.
I20260927 07:05:22.890497 1762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:22.891593 1762 log.cc:824] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:22.894515 1762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87: No bootstrap required, opened a new log
Sep 27 07:05:22 dist-test-slave-p6hz krb5kdc[1682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 07:05:22.903579 1762 raft_consensus.cc:348] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7a83c0db64b57bc3da65f72521c87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46415 } }
I20260927 07:05:22.904030 1762 raft_consensus.cc:374] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:22.904126 1762 raft_consensus.cc:729] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab7a83c0db64b57bc3da65f72521c87, State: Initialized, Role: FOLLOWER
I20260927 07:05:22.904846 1762 consensus_queue.cc:260] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7a83c0db64b57bc3da65f72521c87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46415 } }
I20260927 07:05:22.905077 1762 raft_consensus.cc:388] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:22.905226 1762 raft_consensus.cc:482] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:22.905396 1762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:22.907547 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:22.885888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37571 (local address 127.0.0.1:46415)
0927 07:05:22.886684 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:22.886697 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:22.886737 (+ 40us) server_negotiation.cc:407] Connection header received
0927 07:05:22.888533 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:22.888556 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:22.888966 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:22.889325 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:22.890703 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:22.892129 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:22.893750 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:22.894389 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:22.894512 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:22.899461 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:22.899469 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:22.899490 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:22.899538 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:22.902654 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:22.903775 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:22.903783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:22.903790 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:22.904036 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:22.904574 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:22.904583 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:22.904587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:22.905238 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:22.905579 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:22.906032 (+ 453us) server_negotiation.cc:299] Negotiation successful
0927 07:05:22.906389 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":360,"threads_started":1}
I20260927 07:05:22.908078 1762 raft_consensus.cc:504] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7a83c0db64b57bc3da65f72521c87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46415 } }
I20260927 07:05:22.908573 1762 leader_election.cc:302] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eab7a83c0db64b57bc3da65f72521c87; no voters:
I20260927 07:05:22.910379 1762 leader_election.cc:288] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:22.910472 1767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:22.911197 1767 raft_consensus.cc:686] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [term 1 LEADER]: Becoming Leader. State: Replica: eab7a83c0db64b57bc3da65f72521c87, State: Running, Role: LEADER
I20260927 07:05:22.912272 1767 consensus_queue.cc:237] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7a83c0db64b57bc3da65f72521c87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46415 } }
I20260927 07:05:22.912976 1762 sys_catalog.cc:564] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:22.913288 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:22.915115 1768 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eab7a83c0db64b57bc3da65f72521c87. Latest consensus state: current_term: 1 leader_uuid: "eab7a83c0db64b57bc3da65f72521c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7a83c0db64b57bc3da65f72521c87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46415 } } }
I20260927 07:05:22.915220 1769 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eab7a83c0db64b57bc3da65f72521c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab7a83c0db64b57bc3da65f72521c87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46415 } } }
I20260927 07:05:22.915611 1769 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:22.915611 1768 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:22.928187 1772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:22.934717 1772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:22.947314 1772 catalog_manager.cc:1347] Generated new cluster ID: 1602b09edc044673b3b16a21976ea39e
I20260927 07:05:22.947448 1772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:22.964782 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:22.943159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40383 (local address 127.0.0.1:46415)
0927 07:05:22.943360 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:22.943367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:22.951978 (+ 8611us) server_negotiation.cc:407] Connection header received
0927 07:05:22.952177 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:22.952181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:22.952252 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:22.952344 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:22.953829 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:22.954612 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:22.955929 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:22.956389 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:22.956504 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:22.958812 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:22.958817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:22.958820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:22.958854 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:22.961639 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:22.962470 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:22.962474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:22.962476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:22.962637 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:22.963106 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:22.963110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:22.963112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:22.963513 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:22.963817 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:22.964192 (+ 375us) server_negotiation.cc:299] Negotiation successful
0927 07:05:22.964396 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 07:05:22.984587 1772 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:22.986588 1772 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:23.003602 1772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eab7a83c0db64b57bc3da65f72521c87: Generated new TSK 0
I20260927 07:05:23.004827 1772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:23.011365 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 1695
I20260927 07:05:23.012764 1757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:23.107250 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 1695
W20260927 07:05:23.122015 1680 connection.cc:535] client connection to 127.0.0.1:46415 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46415): Connection reset by peer (error 104)
2026-09-27T07:05:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1961 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:23 dist-test-slave-p6hz krb5kdc[1794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:23 dist-test-slave-p6hz krb5kdc[1794](info): set up 2 sockets
Sep 27 07:05:23 dist-test-slave-p6hz krb5kdc[1794](info): commencing operation
krb5kdc: starting...
W20260927 07:05:24.131541 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:24 dist-test-slave-p6hz krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:24.379942 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44983
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:24.719614 1807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:24.719928 1807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:24.720018 1807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:24.727564 1807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:24.727694 1807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:24.727739 1807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:24.727797 1807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:24.727840 1807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:24.738992 1807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44983
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:24.741191 1807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:24.743008 1807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:24.755460 1812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:24.755489 1815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:24.755528 1813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:24.756781 1807 server_base.cc:1034] running on GCE node
I20260927 07:05:24.757670 1807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:24.759192 1807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:24.760432 1807 hybrid_clock.cc:648] HybridClock initialized: now 1790492724760372 us; error 51 us; skew 500 ppm
I20260927 07:05:24.760794 1807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:24 dist-test-slave-p6hz krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:24.764014 1807 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:24.765582 1807 webserver.cc:460] Webserver started at http://127.0.0.1:44703/ using document root <none> and password file <none>
I20260927 07:05:24.766268 1807 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:24.766388 1807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:24.766683 1807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:24.769780 1807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "67e60b47c97b49a3905a0032ca97be0e"
format_stamp: "Formatted at 2026-09-27 07:05:24 on dist-test-slave-p6hz"
server_key: "aa27f240986918982976c0b205414520"
server_key_iv: "a03c5e1fe15497e12db22d8f58e7fc8a"
server_key_version: "encryptionkey@0"
I20260927 07:05:24.770589 1807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "67e60b47c97b49a3905a0032ca97be0e"
format_stamp: "Formatted at 2026-09-27 07:05:24 on dist-test-slave-p6hz"
server_key: "aa27f240986918982976c0b205414520"
server_key_iv: "a03c5e1fe15497e12db22d8f58e7fc8a"
server_key_version: "encryptionkey@0"
I20260927 07:05:24.776314 1807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260927 07:05:24.779982 1822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:24.781386 1807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:24.781572 1807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "67e60b47c97b49a3905a0032ca97be0e"
format_stamp: "Formatted at 2026-09-27 07:05:24 on dist-test-slave-p6hz"
server_key: "aa27f240986918982976c0b205414520"
server_key_iv: "a03c5e1fe15497e12db22d8f58e7fc8a"
server_key_version: "encryptionkey@0"
I20260927 07:05:24.781778 1807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:24.806754 1807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:24.811456 1807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:24.811743 1807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:24.830024 1807 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44983
I20260927 07:05:24.830046 1873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44983 every 8 connection(s)
I20260927 07:05:24.831404 1807 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:24.835125 1874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:24.835192 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 1807
I20260927 07:05:24.835361 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:24.835786 13799 external_mini_cluster.cc:1428] Setting key 800dd86ab24332b2035cea982f6b6f0a
I20260927 07:05:24.848876 1874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e: Bootstrap starting.
I20260927 07:05:24.849288 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:24.837422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54333 (local address 127.0.0.1:44983)
0927 07:05:24.838230 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:24.838245 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:24.838286 (+ 41us) server_negotiation.cc:407] Connection header received
0927 07:05:24.840025 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:24.840050 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:24.840512 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:24.840856 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:24.841531 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:24.842873 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:24.844683 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:24.845412 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:24.845568 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:24.846901 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:24.846909 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:24.846929 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:24.847564 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:24.847657 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:24.847684 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:24.847813 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:24.848062 (+ 249us) server_negotiation.cc:299] Negotiation successful
0927 07:05:24.848241 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":353,"threads_started":1}
I20260927 07:05:24.852484 1874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:24.853231 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:24.853595 1874 log.cc:824] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:24.856398 1874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e: No bootstrap required, opened a new log
I20260927 07:05:24.864048 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:24.857480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54245 (local address 127.0.0.1:44983)
0927 07:05:24.857702 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:24.857708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:24.857968 (+ 260us) server_negotiation.cc:407] Connection header received
0927 07:05:24.858140 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:24.858145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:24.858222 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:24.858304 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:24.858848 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:24.859666 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:24.861513 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:24.861991 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:24.862087 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:24.863295 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:24.863300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:24.863303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:24.863321 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:24.863361 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:24.863376 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:24.863458 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:24.863636 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 07:05:24.863702 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260927 07:05:24.865720 1874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e60b47c97b49a3905a0032ca97be0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44983 } }
I20260927 07:05:24.866142 1874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:24.866248 1874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67e60b47c97b49a3905a0032ca97be0e, State: Initialized, Role: FOLLOWER
I20260927 07:05:24.866799 1874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e60b47c97b49a3905a0032ca97be0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44983 } }
I20260927 07:05:24.867029 1874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:24.867194 1874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:24.867352 1874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:24.869511 1874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e60b47c97b49a3905a0032ca97be0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44983 } }
I20260927 07:05:24.870028 1874 leader_election.cc:302] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67e60b47c97b49a3905a0032ca97be0e; no voters:
I20260927 07:05:24.870687 1874 leader_election.cc:288] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:24.870747 1884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:24.871506 1884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [term 1 LEADER]: Becoming Leader. State: Replica: 67e60b47c97b49a3905a0032ca97be0e, State: Running, Role: LEADER
I20260927 07:05:24.872076 1884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e60b47c97b49a3905a0032ca97be0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44983 } }
I20260927 07:05:24.872822 1874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:24.874810 1886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67e60b47c97b49a3905a0032ca97be0e. Latest consensus state: current_term: 1 leader_uuid: "67e60b47c97b49a3905a0032ca97be0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e60b47c97b49a3905a0032ca97be0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44983 } } }
I20260927 07:05:24.875299 1886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:24.874886 1885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67e60b47c97b49a3905a0032ca97be0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e60b47c97b49a3905a0032ca97be0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44983 } } }
I20260927 07:05:24.875767 1885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:24.881357 1889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:24.887965 1889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:24.896948 1889 catalog_manager.cc:1347] Generated new cluster ID: 330da891b8674a67ab183ce6d7d2d236
I20260927 07:05:24.897066 1889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:24.920136 1889 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:24.921416 1889 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:24.936215 1889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67e60b47c97b49a3905a0032ca97be0e: Generated new TSK 0
I20260927 07:05:24.937083 1889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:24.952077 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 1807
W20260927 07:05:24.953763 1869 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54333
W20260927 07:05:24.954530 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 1807: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:24.954689 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 1807
W20260927 07:05:24.968295 1792 connection.cc:535] client connection to 127.0.0.1:44983 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44983): Connection reset by peer (error 104)
2026-09-27T07:05:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1845 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:25 dist-test-slave-p6hz krb5kdc[1905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:25 dist-test-slave-p6hz krb5kdc[1905](info): set up 2 sockets
Sep 27 07:05:25 dist-test-slave-p6hz krb5kdc[1905](info): commencing operation
krb5kdc: starting...
W20260927 07:05:25.967448 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:26 dist-test-slave-p6hz krb5kdc[1905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:26.212852 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37893
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:26.540666 1918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:26.541003 1918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:26.541126 1918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:26.548815 1918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:26.548947 1918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:26.548998 1918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:26.549065 1918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:26.549126 1918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:26.559346 1918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37893
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:26.560688 1918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:26.561988 1918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:26.573768 1924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:26.573861 1923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:26.573817 1926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:26.574419 1918 server_base.cc:1034] running on GCE node
I20260927 07:05:26.575338 1918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:26.576810 1918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:26.578047 1918 hybrid_clock.cc:648] HybridClock initialized: now 1790492726578006 us; error 68 us; skew 500 ppm
I20260927 07:05:26.578435 1918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:26 dist-test-slave-p6hz krb5kdc[1905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:26.587381 1918 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:26.589026 1918 webserver.cc:460] Webserver started at http://127.0.0.1:43407/ using document root <none> and password file <none>
I20260927 07:05:26.589756 1918 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:26.589867 1918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:26.590168 1918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:26.593302 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "9397bb13883e4465b49cb22efc40bedd"
format_stamp: "Formatted at 2026-09-27 07:05:26 on dist-test-slave-p6hz"
server_key: "8b9609f850ee755849e9f93e87366fe3"
server_key_iv: "95c45b1d6647cb60064bd074abe75d42"
server_key_version: "encryptionkey@0"
I20260927 07:05:26.594105 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "9397bb13883e4465b49cb22efc40bedd"
format_stamp: "Formatted at 2026-09-27 07:05:26 on dist-test-slave-p6hz"
server_key: "8b9609f850ee755849e9f93e87366fe3"
server_key_iv: "95c45b1d6647cb60064bd074abe75d42"
server_key_version: "encryptionkey@0"
I20260927 07:05:26.599858 1918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260927 07:05:26.603636 1933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:26.605202 1918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:05:26.605409 1918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "9397bb13883e4465b49cb22efc40bedd"
format_stamp: "Formatted at 2026-09-27 07:05:26 on dist-test-slave-p6hz"
server_key: "8b9609f850ee755849e9f93e87366fe3"
server_key_iv: "95c45b1d6647cb60064bd074abe75d42"
server_key_version: "encryptionkey@0"
I20260927 07:05:26.605698 1918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:26.612602 1918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:26.616788 1918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:26.617040 1918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:26.634704 1918 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37893
I20260927 07:05:26.634759 1984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37893 every 8 connection(s)
I20260927 07:05:26.636189 1918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:26.636441 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 1918
I20260927 07:05:26.636636 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:26.637172 13799 external_mini_cluster.cc:1428] Setting key a1bc23d27ac45f7263c3d314ad1c45c9
I20260927 07:05:26.641007 1985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:26.645323 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:26.639092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48545 (local address 127.0.0.1:37893)
0927 07:05:26.640054 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:26.640066 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:26.640110 (+ 44us) server_negotiation.cc:407] Connection header received
0927 07:05:26.641815 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:26.641839 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:26.642191 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:26.642530 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:26.643694 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:26.643701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:26.643717 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:26.643743 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:26.643822 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:26.643846 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:26.644028 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:26.644132 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 07:05:26.644351 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":466,"threads_started":1}
I20260927 07:05:26.649320 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:26.654271 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd: Bootstrap starting.
I20260927 07:05:26.656888 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:26.654538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41561 (local address 127.0.0.1:37893)
0927 07:05:26.654738 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:26.654745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:26.655035 (+ 290us) server_negotiation.cc:407] Connection header received
0927 07:05:26.655243 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:26.655247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:26.655299 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:26.655376 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:26.656292 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:26.656296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:26.656299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:26.656313 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:26.656353 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:26.656372 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:26.656462 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:26.656575 (+ 113us) server_negotiation.cc:299] Negotiation successful
0927 07:05:26.656622 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 07:05:26.658191 1985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:26.659294 1985 log.cc:824] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:26.662158 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd: No bootstrap required, opened a new log
I20260927 07:05:26.669704 1985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9397bb13883e4465b49cb22efc40bedd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260927 07:05:26.670150 1985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:26.670248 1985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9397bb13883e4465b49cb22efc40bedd, State: Initialized, Role: FOLLOWER
I20260927 07:05:26.670915 1985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9397bb13883e4465b49cb22efc40bedd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260927 07:05:26.671236 1985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:26.671439 1985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:26.671662 1985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:26.673925 1985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9397bb13883e4465b49cb22efc40bedd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260927 07:05:26.674484 1985 leader_election.cc:302] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9397bb13883e4465b49cb22efc40bedd; no voters:
I20260927 07:05:26.675107 1985 leader_election.cc:288] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:26.675307 1995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:26.675984 1995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [term 1 LEADER]: Becoming Leader. State: Replica: 9397bb13883e4465b49cb22efc40bedd, State: Running, Role: LEADER
I20260927 07:05:26.676586 1995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9397bb13883e4465b49cb22efc40bedd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260927 07:05:26.677269 1985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:26.679497 1997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9397bb13883e4465b49cb22efc40bedd. Latest consensus state: current_term: 1 leader_uuid: "9397bb13883e4465b49cb22efc40bedd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9397bb13883e4465b49cb22efc40bedd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } } }
I20260927 07:05:26.680016 1997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:26.680661 1996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9397bb13883e4465b49cb22efc40bedd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9397bb13883e4465b49cb22efc40bedd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } } }
I20260927 07:05:26.680944 1996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:26.681061 2004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:26.686182 2004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:26.694779 2004 catalog_manager.cc:1347] Generated new cluster ID: db41e83f931e49e89e8126cf31a092f9
I20260927 07:05:26.694906 2004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:26.709919 2004 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:26.710507 2004 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:26.734870 2004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9397bb13883e4465b49cb22efc40bedd: Generated new TSK 0
I20260927 07:05:26.735813 2004 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:26.747964 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 1918
W20260927 07:05:26.749219 1980 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48545
W20260927 07:05:26.750171 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 1918: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:26.750320 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 1918
2026-09-27T07:05:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1796 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:26 dist-test-slave-p6hz krb5kdc[2016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:26 dist-test-slave-p6hz krb5kdc[2016](info): set up 2 sockets
Sep 27 07:05:26 dist-test-slave-p6hz krb5kdc[2016](info): commencing operation
krb5kdc: starting...
W20260927 07:05:27.779408 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:27 dist-test-slave-p6hz krb5kdc[2016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:28.022174 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33171
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:28.355350 2029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:28.355686 2029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:28.355821 2029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:28.363242 2029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:28.363379 2029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:28.363458 2029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:28.363546 2029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:28.363610 2029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:28.373824 2029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:28.375201 2029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:28.376544 2029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:28.388616 2034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:28.388677 2037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:28.388666 2035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:28.389485 2029 server_base.cc:1034] running on GCE node
I20260927 07:05:28.390266 2029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:28.391747 2029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:28.392957 2029 hybrid_clock.cc:648] HybridClock initialized: now 1790492728392904 us; error 40 us; skew 500 ppm
I20260927 07:05:28.393319 2029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:28 dist-test-slave-p6hz krb5kdc[2016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492728, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:28.396385 2029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:28.397913 2029 webserver.cc:460] Webserver started at http://127.0.0.1:38119/ using document root <none> and password file <none>
I20260927 07:05:28.398581 2029 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:28.398693 2029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:28.398948 2029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:28.402117 2029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "d36b0b55310244f3b9e2a99afe979f5e"
format_stamp: "Formatted at 2026-09-27 07:05:28 on dist-test-slave-p6hz"
server_key: "7606d775adf06b9dc627ffe5f143280a"
server_key_iv: "afdc1e9458ce8280ee4269c4ec9f97aa"
server_key_version: "encryptionkey@0"
I20260927 07:05:28.402885 2029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "d36b0b55310244f3b9e2a99afe979f5e"
format_stamp: "Formatted at 2026-09-27 07:05:28 on dist-test-slave-p6hz"
server_key: "7606d775adf06b9dc627ffe5f143280a"
server_key_iv: "afdc1e9458ce8280ee4269c4ec9f97aa"
server_key_version: "encryptionkey@0"
I20260927 07:05:28.408289 2029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:05:28.411854 2044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:28.413159 2029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:28.413317 2029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "d36b0b55310244f3b9e2a99afe979f5e"
format_stamp: "Formatted at 2026-09-27 07:05:28 on dist-test-slave-p6hz"
server_key: "7606d775adf06b9dc627ffe5f143280a"
server_key_iv: "afdc1e9458ce8280ee4269c4ec9f97aa"
server_key_version: "encryptionkey@0"
I20260927 07:05:28.413501 2029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:28.435395 2029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:28.439714 2029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:28.439986 2029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:28.458127 2029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33171
I20260927 07:05:28.458156 2095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33171 every 8 connection(s)
I20260927 07:05:28.459556 2029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:28.463419 2096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:28.466159 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 2029
I20260927 07:05:28.466378 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:28.466890 13799 external_mini_cluster.cc:1428] Setting key 5c2cfd5f87da41b7ec0dd5cfdb690220
I20260927 07:05:28.475714 2096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e: Bootstrap starting.
I20260927 07:05:28.479614 2096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:28.480947 2096 log.cc:824] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e: Log is configured to *not* fsync() on all Append() calls
Sep 27 07:05:28 dist-test-slave-p6hz krb5kdc[2016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 07:05:28.483994 2096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e: No bootstrap required, opened a new log
I20260927 07:05:28.490536 2099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:28.468801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33571 (local address 127.0.0.1:33171)
0927 07:05:28.469600 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:28.469611 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:28.469654 (+ 43us) server_negotiation.cc:407] Connection header received
0927 07:05:28.471340 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:28.471364 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:28.471795 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:28.472128 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:28.473777 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:28.475109 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:28.476869 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:28.477564 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:28.477702 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:28.482491 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:28.482499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:28.482515 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:28.482550 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:28.486052 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:28.486687 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:28.486697 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:28.486710 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:28.486924 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:28.487287 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:28.487292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:28.487296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:28.488004 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:28.488315 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:28.488695 (+ 380us) server_negotiation.cc:299] Negotiation successful
0927 07:05:28.489149 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":376,"threads_started":1}
I20260927 07:05:28.493532 2096 raft_consensus.cc:348] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36b0b55310244f3b9e2a99afe979f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33171 } }
I20260927 07:05:28.493877 2096 raft_consensus.cc:374] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:28.493984 2096 raft_consensus.cc:729] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36b0b55310244f3b9e2a99afe979f5e, State: Initialized, Role: FOLLOWER
I20260927 07:05:28.494135 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:28.494606 2096 consensus_queue.cc:260] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36b0b55310244f3b9e2a99afe979f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33171 } }
I20260927 07:05:28.494889 2096 raft_consensus.cc:388] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:28.495018 2096 raft_consensus.cc:482] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:28.495189 2096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:28.498394 2096 raft_consensus.cc:504] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36b0b55310244f3b9e2a99afe979f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33171 } }
I20260927 07:05:28.499030 2096 leader_election.cc:302] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d36b0b55310244f3b9e2a99afe979f5e; no voters:
I20260927 07:05:28.499750 2096 leader_election.cc:288] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:28.499832 2106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:28.503396 2106 raft_consensus.cc:686] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [term 1 LEADER]: Becoming Leader. State: Replica: d36b0b55310244f3b9e2a99afe979f5e, State: Running, Role: LEADER
I20260927 07:05:28.504158 2106 consensus_queue.cc:237] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36b0b55310244f3b9e2a99afe979f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33171 } }
I20260927 07:05:28.505601 2096 sys_catalog.cc:564] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:28.513005 2099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:28.499178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60161 (local address 127.0.0.1:33171)
0927 07:05:28.499363 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:28.499369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:28.499738 (+ 369us) server_negotiation.cc:407] Connection header received
0927 07:05:28.499925 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:28.499931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:28.500012 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:28.500112 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:28.501497 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:28.502237 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:28.503577 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:28.504245 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:28.504380 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:28.506668 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:28.506673 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:28.506676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:28.506709 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:28.509616 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:28.510298 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:28.510302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:28.510304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:28.510479 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:28.510872 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:28.510877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:28.510881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:28.511279 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:28.512329 (+ 1050us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:28.512453 (+ 124us) server_negotiation.cc:299] Negotiation successful
0927 07:05:28.512623 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:05:28.514029 2107 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d36b0b55310244f3b9e2a99afe979f5e. Latest consensus state: current_term: 1 leader_uuid: "d36b0b55310244f3b9e2a99afe979f5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36b0b55310244f3b9e2a99afe979f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33171 } } }
I20260927 07:05:28.514492 2107 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:28.515125 2108 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d36b0b55310244f3b9e2a99afe979f5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36b0b55310244f3b9e2a99afe979f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33171 } } }
I20260927 07:05:28.515456 2108 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:28.524354 2115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:28.528798 2115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:28.537684 2115 catalog_manager.cc:1347] Generated new cluster ID: e80901370d1c43dc9642d501b5c85e35
I20260927 07:05:28.537812 2115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:28.550992 2115 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:28.552367 2115 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:28.561172 2115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d36b0b55310244f3b9e2a99afe979f5e: Generated new TSK 0
I20260927 07:05:28.562044 2115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:28.571516 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2029
I20260927 07:05:28.572845 2091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:28.665112 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 2029
2026-09-27T07:05:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1913 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:28 dist-test-slave-p6hz krb5kdc[2129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:28 dist-test-slave-p6hz krb5kdc[2129](info): set up 2 sockets
Sep 27 07:05:28 dist-test-slave-p6hz krb5kdc[2129](info): commencing operation
krb5kdc: starting...
W20260927 07:05:29.679456 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.004s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:29 dist-test-slave-p6hz krb5kdc[2129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:29.926923 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40453
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:30.252674 2142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:30.252986 2142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:30.253108 2142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:30.260489 2142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:30.260604 2142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:30.260676 2142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:30.260757 2142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:30.260815 2142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:30.270805 2142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:30.272112 2142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:30.273403 2142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:30.285192 2147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:30.285252 2150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:30.285220 2148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:30.286144 2142 server_base.cc:1034] running on GCE node
I20260927 07:05:30.286976 2142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:30.288518 2142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:30.289724 2142 hybrid_clock.cc:648] HybridClock initialized: now 1790492730289677 us; error 40 us; skew 500 ppm
I20260927 07:05:30.290091 2142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:30 dist-test-slave-p6hz krb5kdc[2129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492730, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:30.293195 2142 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:30.294732 2142 webserver.cc:460] Webserver started at http://127.0.0.1:34825/ using document root <none> and password file <none>
I20260927 07:05:30.295498 2142 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:30.295624 2142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:30.295918 2142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:30.298995 2142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "396936503a1c4c91ba019058718141ea"
format_stamp: "Formatted at 2026-09-27 07:05:30 on dist-test-slave-p6hz"
server_key: "0b41d64651b697c9e00012092a2f678f"
server_key_iv: "3ea4a7843e6ec5a027eb4d4add9f72c2"
server_key_version: "encryptionkey@0"
I20260927 07:05:30.299844 2142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "396936503a1c4c91ba019058718141ea"
format_stamp: "Formatted at 2026-09-27 07:05:30 on dist-test-slave-p6hz"
server_key: "0b41d64651b697c9e00012092a2f678f"
server_key_iv: "3ea4a7843e6ec5a027eb4d4add9f72c2"
server_key_version: "encryptionkey@0"
I20260927 07:05:30.305310 2142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260927 07:05:30.309273 2157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:30.310714 2142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:05:30.310891 2142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "396936503a1c4c91ba019058718141ea"
format_stamp: "Formatted at 2026-09-27 07:05:30 on dist-test-slave-p6hz"
server_key: "0b41d64651b697c9e00012092a2f678f"
server_key_iv: "3ea4a7843e6ec5a027eb4d4add9f72c2"
server_key_version: "encryptionkey@0"
I20260927 07:05:30.311087 2142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:30.324100 2142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:30.328269 2142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:30.328534 2142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:30.346767 2142 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40453
I20260927 07:05:30.346813 2208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40453 every 8 connection(s)
I20260927 07:05:30.348176 2142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:30.349680 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 2142
I20260927 07:05:30.349838 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:30.350234 13799 external_mini_cluster.cc:1428] Setting key 216bfc6c7b9cbde3ca2a382300054da5
I20260927 07:05:30.352406 2209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:30.365181 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:30.351991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57753 (local address 127.0.0.1:40453)
0927 07:05:30.352797 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:30.352812 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:30.352861 (+ 49us) server_negotiation.cc:407] Connection header received
0927 07:05:30.355098 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:30.355128 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:30.355675 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:30.356121 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:30.356620 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:30.358620 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:30.360019 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:30.361024 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:30.361195 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:30.362267 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:30.362277 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:30.362295 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:30.363056 (+ 761us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:30.363180 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:30.363214 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:30.363341 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:30.363596 (+ 255us) server_negotiation.cc:299] Negotiation successful
0927 07:05:30.363884 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":332,"threads_started":1}
I20260927 07:05:30.366325 2209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea: Bootstrap starting.
I20260927 07:05:30.368785 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:30.369681 2209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:30.370854 2209 log.cc:824] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:30.373749 2209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea: No bootstrap required, opened a new log
I20260927 07:05:30.380334 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:30.373745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42827 (local address 127.0.0.1:40453)
0927 07:05:30.373930 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:30.373937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:30.374294 (+ 357us) server_negotiation.cc:407] Connection header received
0927 07:05:30.374497 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:30.374501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:30.374561 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:30.374638 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:30.375252 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:30.376059 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:30.377915 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:30.378371 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:30.378480 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:30.379623 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:30.379631 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:30.379634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:30.379653 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:30.379691 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:30.379710 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:30.379785 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:30.379913 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 07:05:30.380001 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 07:05:30.383293 2209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396936503a1c4c91ba019058718141ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40453 } }
I20260927 07:05:30.383728 2209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:30.383839 2209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396936503a1c4c91ba019058718141ea, State: Initialized, Role: FOLLOWER
I20260927 07:05:30.384413 2209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396936503a1c4c91ba019058718141ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40453 } }
I20260927 07:05:30.384622 2209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:30.384749 2209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:30.384902 2209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:30.387055 2209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396936503a1c4c91ba019058718141ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40453 } }
I20260927 07:05:30.387627 2209 leader_election.cc:302] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 396936503a1c4c91ba019058718141ea; no voters:
I20260927 07:05:30.388207 2209 leader_election.cc:288] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:30.388350 2219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:30.389024 2219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [term 1 LEADER]: Becoming Leader. State: Replica: 396936503a1c4c91ba019058718141ea, State: Running, Role: LEADER
I20260927 07:05:30.389624 2219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396936503a1c4c91ba019058718141ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40453 } }
I20260927 07:05:30.390197 2209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:30.392536 2220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "396936503a1c4c91ba019058718141ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396936503a1c4c91ba019058718141ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40453 } } }
I20260927 07:05:30.392489 2221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 396936503a1c4c91ba019058718141ea. Latest consensus state: current_term: 1 leader_uuid: "396936503a1c4c91ba019058718141ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396936503a1c4c91ba019058718141ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40453 } } }
I20260927 07:05:30.392918 2221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:30.392915 2220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:30.394843 2226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:30.399636 2226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:30.408870 2226 catalog_manager.cc:1347] Generated new cluster ID: 06828a752cc14986b644f5fb4c9b1cea
I20260927 07:05:30.408989 2226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:30.418032 2226 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:30.419924 2226 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:30.435535 2226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 396936503a1c4c91ba019058718141ea: Generated new TSK 0
I20260927 07:05:30.436587 2226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:30.452298 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2142
W20260927 07:05:30.453481 2204 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57753
W20260927 07:05:30.454312 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2142: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:30.454458 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 2142
2026-09-27T07:05:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1789 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:30 dist-test-slave-p6hz krb5kdc[2240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:30 dist-test-slave-p6hz krb5kdc[2240](info): set up 2 sockets
Sep 27 07:05:30 dist-test-slave-p6hz krb5kdc[2240](info): commencing operation
krb5kdc: starting...
W20260927 07:05:31.475406 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:31 dist-test-slave-p6hz krb5kdc[2240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:31.727038 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:32.072156 2253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:32.072465 2253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:32.072595 2253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:32.080259 2253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:32.080402 2253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:32.080479 2253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:32.080565 2253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:32.080611 2253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:32.091037 2253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:32.092388 2253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:32.093662 2253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:32.106565 2259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:32.106585 2261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:32.107403 2258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:32.107411 2253 server_base.cc:1034] running on GCE node
I20260927 07:05:32.108268 2253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:32.109793 2253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:32.111019 2253 hybrid_clock.cc:648] HybridClock initialized: now 1790492732110955 us; error 57 us; skew 500 ppm
I20260927 07:05:32.111413 2253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:32 dist-test-slave-p6hz krb5kdc[2240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492732, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:32.114496 2253 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:32.116036 2253 webserver.cc:460] Webserver started at http://127.0.0.1:41271/ using document root <none> and password file <none>
I20260927 07:05:32.116711 2253 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:32.116832 2253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:32.117113 2253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:32.120357 2253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "e6d7d2323d524eb4adfb7d05449b02f2"
format_stamp: "Formatted at 2026-09-27 07:05:32 on dist-test-slave-p6hz"
server_key: "e270aab9616c0e3c891cd4285b817877"
server_key_iv: "2b85a1fc2865adebba3b8e1787e5e7f9"
server_key_version: "encryptionkey@0"
I20260927 07:05:32.121181 2253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "e6d7d2323d524eb4adfb7d05449b02f2"
format_stamp: "Formatted at 2026-09-27 07:05:32 on dist-test-slave-p6hz"
server_key: "e270aab9616c0e3c891cd4285b817877"
server_key_iv: "2b85a1fc2865adebba3b8e1787e5e7f9"
server_key_version: "encryptionkey@0"
I20260927 07:05:32.126772 2253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:05:32.130491 2268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:32.132050 2253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 07:05:32.132237 2253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "e6d7d2323d524eb4adfb7d05449b02f2"
format_stamp: "Formatted at 2026-09-27 07:05:32 on dist-test-slave-p6hz"
server_key: "e270aab9616c0e3c891cd4285b817877"
server_key_iv: "2b85a1fc2865adebba3b8e1787e5e7f9"
server_key_version: "encryptionkey@0"
I20260927 07:05:32.132438 2253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:32.138332 2253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:32.142992 2253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:32.143330 2253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:32.162730 2253 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41715
I20260927 07:05:32.162782 2319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41715 every 8 connection(s)
I20260927 07:05:32.164202 2253 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:32.168162 2320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:32.171106 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 2253
I20260927 07:05:32.171308 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:32.171803 13799 external_mini_cluster.cc:1428] Setting key c85a80934b462416a336fe0271ab525d
I20260927 07:05:32.180282 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:32.173693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47093 (local address 127.0.0.1:41715)
0927 07:05:32.174485 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:32.174501 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:32.174546 (+ 45us) server_negotiation.cc:407] Connection header received
0927 07:05:32.176506 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:32.176535 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:32.176965 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:32.177346 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:32.178531 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:32.178539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:32.178553 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:32.178574 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:32.178645 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:32.178668 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:32.178864 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:32.179016 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 07:05:32.179288 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":362,"threads_started":1}
I20260927 07:05:32.180804 2320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2: Bootstrap starting.
I20260927 07:05:32.184154 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:32.184156 2320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:32.185428 2320 log.cc:824] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:32.189275 2320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2: No bootstrap required, opened a new log
I20260927 07:05:32.192018 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:32.189468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42573 (local address 127.0.0.1:41715)
0927 07:05:32.189680 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:32.189688 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:32.190137 (+ 449us) server_negotiation.cc:407] Connection header received
0927 07:05:32.190303 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:32.190307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:32.190356 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:32.190441 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:32.191349 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:32.191353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:32.191356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:32.191372 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:32.191412 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:32.191428 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:32.191542 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:32.191639 (+ 97us) server_negotiation.cc:299] Negotiation successful
0927 07:05:32.191709 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 07:05:32.197090 2320 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d7d2323d524eb4adfb7d05449b02f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41715 } }
I20260927 07:05:32.197459 2320 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:32.197567 2320 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d7d2323d524eb4adfb7d05449b02f2, State: Initialized, Role: FOLLOWER
I20260927 07:05:32.198145 2320 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d7d2323d524eb4adfb7d05449b02f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41715 } }
I20260927 07:05:32.198385 2320 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:32.198515 2320 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:32.198805 2320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:32.201323 2320 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d7d2323d524eb4adfb7d05449b02f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41715 } }
I20260927 07:05:32.201821 2320 leader_election.cc:302] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6d7d2323d524eb4adfb7d05449b02f2; no voters:
I20260927 07:05:32.202394 2320 leader_election.cc:288] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:32.202536 2330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:32.203198 2330 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [term 1 LEADER]: Becoming Leader. State: Replica: e6d7d2323d524eb4adfb7d05449b02f2, State: Running, Role: LEADER
I20260927 07:05:32.203727 2330 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d7d2323d524eb4adfb7d05449b02f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41715 } }
I20260927 07:05:32.204521 2320 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:32.206549 2331 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6d7d2323d524eb4adfb7d05449b02f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d7d2323d524eb4adfb7d05449b02f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41715 } } }
I20260927 07:05:32.206568 2332 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6d7d2323d524eb4adfb7d05449b02f2. Latest consensus state: current_term: 1 leader_uuid: "e6d7d2323d524eb4adfb7d05449b02f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d7d2323d524eb4adfb7d05449b02f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41715 } } }
I20260927 07:05:32.206990 2331 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:32.207162 2332 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:32.208591 2336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:32.214187 2336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:32.223435 2336 catalog_manager.cc:1347] Generated new cluster ID: 8fa3366d3ca94e67a364f6d0b8f20c5a
I20260927 07:05:32.223577 2336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:32.262391 2336 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:32.262917 2336 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:32.271836 2336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6d7d2323d524eb4adfb7d05449b02f2: Generated new TSK 0
I20260927 07:05:32.272894 2336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:32.289911 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2253
W20260927 07:05:32.291118 2315 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47093
W20260927 07:05:32.291880 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2253: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:32.292017 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 2253
2026-09-27T07:05:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1837 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:32 dist-test-slave-p6hz krb5kdc[2351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:32 dist-test-slave-p6hz krb5kdc[2351](info): set up 2 sockets
Sep 27 07:05:32 dist-test-slave-p6hz krb5kdc[2351](info): commencing operation
krb5kdc: starting...
W20260927 07:05:33.311422 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:33 dist-test-slave-p6hz krb5kdc[2351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:33.562387 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:33.885702 2364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:33.886026 2364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:33.886166 2364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:33.893529 2364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:33.893666 2364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:33.893738 2364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:33.893816 2364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:33.893874 2364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:33.904163 2364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:33.905498 2364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:33.906775 2364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:33.918891 2369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:33.918857 2370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:33.918928 2372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:33.919804 2364 server_base.cc:1034] running on GCE node
I20260927 07:05:33.920655 2364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:33.922094 2364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:33.923317 2364 hybrid_clock.cc:648] HybridClock initialized: now 1790492733923266 us; error 46 us; skew 500 ppm
I20260927 07:05:33.923688 2364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:33 dist-test-slave-p6hz krb5kdc[2351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:33.926753 2364 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:33.928304 2364 webserver.cc:460] Webserver started at http://127.0.0.1:36431/ using document root <none> and password file <none>
I20260927 07:05:33.928998 2364 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:33.929101 2364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:33.929358 2364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:33.932418 2364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "fb424c9c66ef48e9ac433505ac7cc40b"
format_stamp: "Formatted at 2026-09-27 07:05:33 on dist-test-slave-p6hz"
server_key: "0652d2068ac5f94d02331c8a64cc0667"
server_key_iv: "84030dc292bbea205c63aac9622673a1"
server_key_version: "encryptionkey@0"
I20260927 07:05:33.933177 2364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "fb424c9c66ef48e9ac433505ac7cc40b"
format_stamp: "Formatted at 2026-09-27 07:05:33 on dist-test-slave-p6hz"
server_key: "0652d2068ac5f94d02331c8a64cc0667"
server_key_iv: "84030dc292bbea205c63aac9622673a1"
server_key_version: "encryptionkey@0"
I20260927 07:05:33.938247 2364 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260927 07:05:33.941680 2379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:33.942988 2364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:05:33.943171 2364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "fb424c9c66ef48e9ac433505ac7cc40b"
format_stamp: "Formatted at 2026-09-27 07:05:33 on dist-test-slave-p6hz"
server_key: "0652d2068ac5f94d02331c8a64cc0667"
server_key_iv: "84030dc292bbea205c63aac9622673a1"
server_key_version: "encryptionkey@0"
I20260927 07:05:33.943353 2364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:33.957803 2364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:33.961812 2364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:33.962071 2364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:33.978744 2364 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46715
I20260927 07:05:33.978765 2430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46715 every 8 connection(s)
I20260927 07:05:33.980166 2364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:33.983894 2431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:33.985523 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 2364
I20260927 07:05:33.985708 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:33.986111 13799 external_mini_cluster.cc:1428] Setting key 2c78f82ca0efd367281936a04ee62c4d
I20260927 07:05:33.998034 2431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b: Bootstrap starting.
Sep 27 07:05:33 dist-test-slave-p6hz krb5kdc[2351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 07:05:34.001127 2431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:34.002444 2431 log.cc:824] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:34.005834 2431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b: No bootstrap required, opened a new log
I20260927 07:05:34.009841 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:33.988066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:54213 (local address 127.0.0.1:46715)
0927 07:05:33.988902 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:33.988917 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:33.988972 (+ 55us) server_negotiation.cc:407] Connection header received
0927 07:05:33.990659 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:33.990688 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:33.991298 (+ 610us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:33.991724 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:33.993028 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:33.994973 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:33.996194 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:33.997194 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:33.997359 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:34.001261 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:34.001272 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:34.001292 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:34.001342 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:34.005086 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:34.006015 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:34.006024 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:34.006031 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:34.006239 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:34.006685 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:34.006690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:34.006693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:34.007402 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:34.007743 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:34.008152 (+ 409us) server_negotiation.cc:299] Negotiation successful
0927 07:05:34.008504 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":352,"threads_started":1}
I20260927 07:05:34.014004 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:34.015465 2431 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb424c9c66ef48e9ac433505ac7cc40b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260927 07:05:34.015818 2431 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:34.015908 2431 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb424c9c66ef48e9ac433505ac7cc40b, State: Initialized, Role: FOLLOWER
I20260927 07:05:34.016466 2431 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb424c9c66ef48e9ac433505ac7cc40b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260927 07:05:34.016737 2431 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:34.016876 2431 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:34.017031 2431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:34.019656 2431 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb424c9c66ef48e9ac433505ac7cc40b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260927 07:05:34.020147 2431 leader_election.cc:302] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb424c9c66ef48e9ac433505ac7cc40b; no voters:
I20260927 07:05:34.020694 2431 leader_election.cc:288] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:34.020853 2441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:34.021943 2441 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [term 1 LEADER]: Becoming Leader. State: Replica: fb424c9c66ef48e9ac433505ac7cc40b, State: Running, Role: LEADER
I20260927 07:05:34.022507 2441 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb424c9c66ef48e9ac433505ac7cc40b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260927 07:05:34.023126 2431 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:34.025118 2443 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb424c9c66ef48e9ac433505ac7cc40b. Latest consensus state: current_term: 1 leader_uuid: "fb424c9c66ef48e9ac433505ac7cc40b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb424c9c66ef48e9ac433505ac7cc40b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } } }
I20260927 07:05:34.025509 2443 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:34.025542 2442 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb424c9c66ef48e9ac433505ac7cc40b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb424c9c66ef48e9ac433505ac7cc40b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } } }
I20260927 07:05:34.025837 2442 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:34.026504 2446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:34.036509 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:34.018229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:41817 (local address 127.0.0.1:46715)
0927 07:05:34.018461 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:34.018469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:34.018964 (+ 495us) server_negotiation.cc:407] Connection header received
0927 07:05:34.019089 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:34.019094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:34.019196 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:34.019294 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:34.020783 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:34.022021 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:34.023951 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:34.024674 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:34.024809 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:34.028336 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:34.028344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:34.028349 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:34.028393 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:34.032827 (+ 4434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:34.033736 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:34.033743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:34.033746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:34.033878 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:34.034342 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:34.034348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:34.034352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:34.034735 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:34.035004 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:34.036073 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0927 07:05:34.036235 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 07:05:34.042807 2446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:34.052315 2446 catalog_manager.cc:1347] Generated new cluster ID: f94ab1a2a13c4d029907610b34716712
I20260927 07:05:34.052441 2446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:34.074183 2446 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:34.075623 2446 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:34.084307 2446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb424c9c66ef48e9ac433505ac7cc40b: Generated new TSK 0
I20260927 07:05:34.085184 2446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:34.099164 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2364
I20260927 07:05:34.101790 2426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:34.193568 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 2364
W20260927 07:05:34.207778 2349 connection.cc:535] client connection to 127.0.0.1:46715 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46715): Connection reset by peer (error 104)
2026-09-27T07:05:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1901 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:34 dist-test-slave-p6hz krb5kdc[2463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:34 dist-test-slave-p6hz krb5kdc[2463](info): set up 2 sockets
Sep 27 07:05:34 dist-test-slave-p6hz krb5kdc[2463](info): commencing operation
krb5kdc: starting...
W20260927 07:05:35.219496 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.011s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:35 dist-test-slave-p6hz krb5kdc[2463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:35.470078 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:35.802255 2476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:35.802582 2476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:35.802728 2476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:35.810107 2476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:35.810257 2476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:35.810333 2476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:35.810411 2476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:35.810472 2476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:35.820815 2476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:35.822157 2476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:35.823503 2476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:35.835538 2481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:35.835633 2484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:35.835664 2482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:35.836289 2476 server_base.cc:1034] running on GCE node
I20260927 07:05:35.837049 2476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:35.838574 2476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:35.839774 2476 hybrid_clock.cc:648] HybridClock initialized: now 1790492735839726 us; error 43 us; skew 500 ppm
I20260927 07:05:35.840119 2476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:35 dist-test-slave-p6hz krb5kdc[2463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:35.843163 2476 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:35.844713 2476 webserver.cc:460] Webserver started at http://127.0.0.1:46115/ using document root <none> and password file <none>
I20260927 07:05:35.845384 2476 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:35.845500 2476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:35.845798 2476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:35.848932 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "c7343329422944388ae7c27cab0d7415"
format_stamp: "Formatted at 2026-09-27 07:05:35 on dist-test-slave-p6hz"
server_key: "58173312ca0fa985f45781a9f184bfc5"
server_key_iv: "9f90f7e682ca8665b08fb19e429e7b36"
server_key_version: "encryptionkey@0"
I20260927 07:05:35.849704 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "c7343329422944388ae7c27cab0d7415"
format_stamp: "Formatted at 2026-09-27 07:05:35 on dist-test-slave-p6hz"
server_key: "58173312ca0fa985f45781a9f184bfc5"
server_key_iv: "9f90f7e682ca8665b08fb19e429e7b36"
server_key_version: "encryptionkey@0"
I20260927 07:05:35.854939 2476 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260927 07:05:35.858551 2491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:35.860090 2476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:35.860275 2476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "c7343329422944388ae7c27cab0d7415"
format_stamp: "Formatted at 2026-09-27 07:05:35 on dist-test-slave-p6hz"
server_key: "58173312ca0fa985f45781a9f184bfc5"
server_key_iv: "9f90f7e682ca8665b08fb19e429e7b36"
server_key_version: "encryptionkey@0"
I20260927 07:05:35.860478 2476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:35.896385 2476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:35.900560 2476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:35.900836 2476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:35.918107 2476 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46539
I20260927 07:05:35.918128 2542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46539 every 8 connection(s)
I20260927 07:05:35.919538 2476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:35.923388 2543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:35.924687 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 2476
I20260927 07:05:35.924872 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:35.925395 13799 external_mini_cluster.cc:1428] Setting key 723d1938e02583afde7dab83dbae95ef
I20260927 07:05:35.937435 2543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415: Bootstrap starting.
I20260927 07:05:35.938930 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:35.927350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:56047 (local address 127.0.0.1:46539)
0927 07:05:35.928171 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:35.928183 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:35.928226 (+ 43us) server_negotiation.cc:407] Connection header received
0927 07:05:35.929849 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:35.929872 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:35.930327 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:35.930670 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:35.931295 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:35.932680 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:35.934513 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:35.935284 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:35.935424 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:35.936682 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:35.936690 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:35.936704 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:35.937241 (+ 537us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:35.937320 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:35.937344 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:35.937466 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:35.937759 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 07:05:35.937951 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":353,"threads_started":1}
I20260927 07:05:35.940688 2543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:35.941787 2543 log.cc:824] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:35.942708 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:35.944921 2543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415: No bootstrap required, opened a new log
I20260927 07:05:35.953801 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:35.947524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:55053 (local address 127.0.0.1:46539)
0927 07:05:35.947723 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:35.947729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:35.948054 (+ 325us) server_negotiation.cc:407] Connection header received
0927 07:05:35.948202 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:35.948207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:35.948293 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:35.948400 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:35.948841 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:35.949970 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:35.951238 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:35.951925 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:35.952052 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:35.953015 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:35.953022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:35.953027 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:35.953047 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:35.953096 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:35.953116 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:35.953220 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:35.953441 (+ 221us) server_negotiation.cc:299] Negotiation successful
0927 07:05:35.953494 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 07:05:35.954306 2543 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7343329422944388ae7c27cab0d7415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } }
I20260927 07:05:35.954787 2543 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:35.954895 2543 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7343329422944388ae7c27cab0d7415, State: Initialized, Role: FOLLOWER
I20260927 07:05:35.955516 2543 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7343329422944388ae7c27cab0d7415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } }
I20260927 07:05:35.955739 2543 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:35.955835 2543 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:35.955953 2543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:35.958055 2543 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7343329422944388ae7c27cab0d7415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } }
I20260927 07:05:35.958549 2543 leader_election.cc:302] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7343329422944388ae7c27cab0d7415; no voters:
I20260927 07:05:35.959064 2543 leader_election.cc:288] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:35.959216 2553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:35.959856 2553 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [term 1 LEADER]: Becoming Leader. State: Replica: c7343329422944388ae7c27cab0d7415, State: Running, Role: LEADER
I20260927 07:05:35.960402 2553 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7343329422944388ae7c27cab0d7415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } }
I20260927 07:05:35.961275 2543 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:35.963439 2555 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7343329422944388ae7c27cab0d7415. Latest consensus state: current_term: 1 leader_uuid: "c7343329422944388ae7c27cab0d7415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7343329422944388ae7c27cab0d7415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } } }
I20260927 07:05:35.963831 2555 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:35.964313 2554 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7343329422944388ae7c27cab0d7415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7343329422944388ae7c27cab0d7415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } } }
I20260927 07:05:35.964713 2554 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:35.965715 2562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:35.971290 2562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:35.981621 2562 catalog_manager.cc:1347] Generated new cluster ID: df325174b87c4698ac93485cab1c7736
I20260927 07:05:35.982102 2562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:35.990747 2562 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:35.992120 2562 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:36.002664 2562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7343329422944388ae7c27cab0d7415: Generated new TSK 0
I20260927 07:05:36.003647 2562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:36.017776 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2476
W20260927 07:05:36.018931 2538 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.90:56047
W20260927 07:05:36.019692 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2476: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:36.019815 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 2476
W20260927 07:05:36.033922 2461 connection.cc:535] client connection to 127.0.0.1:46539 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46539): Connection reset by peer (error 104)
2026-09-27T07:05:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1825 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:36 dist-test-slave-p6hz krb5kdc[2575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:36 dist-test-slave-p6hz krb5kdc[2575](info): set up 2 sockets
Sep 27 07:05:36 dist-test-slave-p6hz krb5kdc[2575](info): commencing operation
krb5kdc: starting...
W20260927 07:05:37.043614 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:37 dist-test-slave-p6hz krb5kdc[2575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:37.298659 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:37.627646 2588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:37.627939 2588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:37.628052 2588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:37.635473 2588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:37.635588 2588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:37.635645 2588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:37.635730 2588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:37.635788 2588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:37.647202 2588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:37.648558 2588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:37.649817 2588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:37.661412 2594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:37.661389 2593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:37.661389 2596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:37.662420 2588 server_base.cc:1034] running on GCE node
I20260927 07:05:37.663300 2588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:37.664809 2588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:37.666009 2588 hybrid_clock.cc:648] HybridClock initialized: now 1790492737665967 us; error 40 us; skew 500 ppm
I20260927 07:05:37.666352 2588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:37 dist-test-slave-p6hz krb5kdc[2575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492737, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:37.674574 2588 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:37.676291 2588 webserver.cc:460] Webserver started at http://127.0.0.1:42807/ using document root <none> and password file <none>
I20260927 07:05:37.677006 2588 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:37.677134 2588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:37.677436 2588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:37.680552 2588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "fa0a60eb23304d2fb268bfbe46d440f8"
format_stamp: "Formatted at 2026-09-27 07:05:37 on dist-test-slave-p6hz"
server_key: "955c4552f7ced49968c95602f34ecbb3"
server_key_iv: "99bf6574f9f2217b940471228f44a50e"
server_key_version: "encryptionkey@0"
I20260927 07:05:37.681361 2588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "fa0a60eb23304d2fb268bfbe46d440f8"
format_stamp: "Formatted at 2026-09-27 07:05:37 on dist-test-slave-p6hz"
server_key: "955c4552f7ced49968c95602f34ecbb3"
server_key_iv: "99bf6574f9f2217b940471228f44a50e"
server_key_version: "encryptionkey@0"
I20260927 07:05:37.686617 2588 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260927 07:05:37.690200 2603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:37.691627 2588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:37.691793 2588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "fa0a60eb23304d2fb268bfbe46d440f8"
format_stamp: "Formatted at 2026-09-27 07:05:37 on dist-test-slave-p6hz"
server_key: "955c4552f7ced49968c95602f34ecbb3"
server_key_iv: "99bf6574f9f2217b940471228f44a50e"
server_key_version: "encryptionkey@0"
I20260927 07:05:37.692029 2588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:37.697784 2588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:37.701896 2588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:37.702175 2588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:37.719499 2588 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43055
I20260927 07:05:37.719537 2654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43055 every 8 connection(s)
I20260927 07:05:37.720883 2588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:37.722097 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 2588
I20260927 07:05:37.722267 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:37.722805 13799 external_mini_cluster.cc:1428] Setting key bf766f78dde4feb342e37c28d964e199
I20260927 07:05:37.725466 2655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:37.730319 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:37.724610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:52869 (local address 127.0.0.1:43055)
0927 07:05:37.725380 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:37.725391 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:37.725429 (+ 38us) server_negotiation.cc:407] Connection header received
0927 07:05:37.727060 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:37.727080 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:37.727452 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:37.727818 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:37.728756 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:37.728762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:37.728774 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:37.728792 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:37.728869 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:37.728888 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:37.729121 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:37.729260 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 07:05:37.729428 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":353,"threads_started":1}
I20260927 07:05:37.733819 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:37.736447 2655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8: Bootstrap starting.
I20260927 07:05:37.739852 2655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:37.740571 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:37.738076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:38633 (local address 127.0.0.1:43055)
0927 07:05:37.738256 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:37.738261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:37.738702 (+ 441us) server_negotiation.cc:407] Connection header received
0927 07:05:37.738831 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:37.738835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:37.738899 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:37.739018 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:37.739869 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:37.739874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:37.739879 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:37.739897 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:37.739944 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:37.739965 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:37.740086 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:37.740226 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 07:05:37.740295 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 07:05:37.741153 2655 log.cc:824] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:37.744019 2655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8: No bootstrap required, opened a new log
I20260927 07:05:37.752030 2655 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0a60eb23304d2fb268bfbe46d440f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43055 } }
I20260927 07:05:37.752400 2655 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:37.752475 2655 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0a60eb23304d2fb268bfbe46d440f8, State: Initialized, Role: FOLLOWER
I20260927 07:05:37.753001 2655 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0a60eb23304d2fb268bfbe46d440f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43055 } }
I20260927 07:05:37.753203 2655 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:37.753301 2655 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:37.753424 2655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:37.755652 2655 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0a60eb23304d2fb268bfbe46d440f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43055 } }
I20260927 07:05:37.756230 2655 leader_election.cc:302] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa0a60eb23304d2fb268bfbe46d440f8; no voters:
I20260927 07:05:37.756850 2655 leader_election.cc:288] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:37.756949 2665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:37.757604 2665 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [term 1 LEADER]: Becoming Leader. State: Replica: fa0a60eb23304d2fb268bfbe46d440f8, State: Running, Role: LEADER
I20260927 07:05:37.758116 2665 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0a60eb23304d2fb268bfbe46d440f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43055 } }
I20260927 07:05:37.758848 2655 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:37.760557 2666 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa0a60eb23304d2fb268bfbe46d440f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0a60eb23304d2fb268bfbe46d440f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43055 } } }
I20260927 07:05:37.760869 2666 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:37.761677 2671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:37.761508 2667 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa0a60eb23304d2fb268bfbe46d440f8. Latest consensus state: current_term: 1 leader_uuid: "fa0a60eb23304d2fb268bfbe46d440f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0a60eb23304d2fb268bfbe46d440f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43055 } } }
I20260927 07:05:37.766798 2667 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:37.767477 2671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:37.783089 2671 catalog_manager.cc:1347] Generated new cluster ID: 53c5e637281b4927b05f49b64345e0bd
I20260927 07:05:37.783644 2671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:37.793437 2671 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:37.793959 2671 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:37.807972 2671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa0a60eb23304d2fb268bfbe46d440f8: Generated new TSK 0
I20260927 07:05:37.809103 2671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:37.823127 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2588
W20260927 07:05:37.824386 2650 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.90:52869
W20260927 07:05:37.825045 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2588: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:37.825201 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 2588
2026-09-27T07:05:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1805 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:37 dist-test-slave-p6hz krb5kdc[2686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:37 dist-test-slave-p6hz krb5kdc[2686](info): set up 2 sockets
Sep 27 07:05:37 dist-test-slave-p6hz krb5kdc[2686](info): commencing operation
krb5kdc: starting...
W20260927 07:05:38.843458 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:39 dist-test-slave-p6hz krb5kdc[2686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:39.092818 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32953
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:39.420656 2699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:39.420972 2699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:39.421089 2699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:39.428689 2699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:39.428820 2699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:39.428879 2699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:39.428964 2699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:39.429025 2699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:39.439360 2699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32953
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:39.440698 2699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:39.442036 2699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:39.453831 2704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:39.453809 2705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:39.454111 2707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:39.454670 2699 server_base.cc:1034] running on GCE node
I20260927 07:05:39.455497 2699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:39.456881 2699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:39.458086 2699 hybrid_clock.cc:648] HybridClock initialized: now 1790492739458036 us; error 35 us; skew 500 ppm
I20260927 07:05:39.458424 2699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:39 dist-test-slave-p6hz krb5kdc[2686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492739, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:39.461580 2699 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:39.463266 2699 webserver.cc:460] Webserver started at http://127.0.0.1:42445/ using document root <none> and password file <none>
I20260927 07:05:39.463963 2699 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:39.464089 2699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:39.464380 2699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:39.467520 2699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "b6d86eece16244288f01e221755902c4"
format_stamp: "Formatted at 2026-09-27 07:05:39 on dist-test-slave-p6hz"
server_key: "439566b06df604b3bf79d9aa17e37a21"
server_key_iv: "068046ab8f28d0fd2728ca58fc60db29"
server_key_version: "encryptionkey@0"
I20260927 07:05:39.468336 2699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "b6d86eece16244288f01e221755902c4"
format_stamp: "Formatted at 2026-09-27 07:05:39 on dist-test-slave-p6hz"
server_key: "439566b06df604b3bf79d9aa17e37a21"
server_key_iv: "068046ab8f28d0fd2728ca58fc60db29"
server_key_version: "encryptionkey@0"
I20260927 07:05:39.473731 2699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260927 07:05:39.477702 2714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:39.479068 2699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:05:39.479288 2699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "b6d86eece16244288f01e221755902c4"
format_stamp: "Formatted at 2026-09-27 07:05:39 on dist-test-slave-p6hz"
server_key: "439566b06df604b3bf79d9aa17e37a21"
server_key_iv: "068046ab8f28d0fd2728ca58fc60db29"
server_key_version: "encryptionkey@0"
I20260927 07:05:39.479471 2699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:39.510679 2699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:39.515300 2699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:39.515588 2699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:39.533025 2699 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32953
I20260927 07:05:39.533062 2765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32953 every 8 connection(s)
I20260927 07:05:39.534402 2699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:39.537431 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 2699
I20260927 07:05:39.537606 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:39.538147 13799 external_mini_cluster.cc:1428] Setting key 69bf4c9a47dc2e999553f3803dc9500b
I20260927 07:05:39.538636 2766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:39.554019 2766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4: Bootstrap starting.
Sep 27 07:05:39 dist-test-slave-p6hz krb5kdc[2686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 07:05:39.558010 2766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:39.559443 2766 log.cc:824] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:39.563079 2766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4: No bootstrap required, opened a new log
I20260927 07:05:39.564453 2769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:39.540850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:43653 (local address 127.0.0.1:32953)
0927 07:05:39.541785 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:39.541803 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:39.541861 (+ 58us) server_negotiation.cc:407] Connection header received
0927 07:05:39.544410 (+ 2549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:39.544447 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:39.545081 (+ 634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:39.545682 (+ 601us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:39.547096 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:39.549032 (+ 1936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:39.550368 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:39.551508 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:39.551705 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:39.555658 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:39.555671 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:39.555693 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:39.555752 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:39.559645 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:39.560680 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:39.560690 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:39.560697 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:39.560856 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:39.561309 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:39.561313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:39.561316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:39.561999 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:39.562296 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:39.562733 (+ 437us) server_negotiation.cc:299] Negotiation successful
0927 07:05:39.563084 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":385,"threads_started":1}
I20260927 07:05:39.568610 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:39.572001 2766 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d86eece16244288f01e221755902c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32953 } }
I20260927 07:05:39.572371 2766 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:39.572476 2766 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d86eece16244288f01e221755902c4, State: Initialized, Role: FOLLOWER
I20260927 07:05:39.573225 2766 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d86eece16244288f01e221755902c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32953 } }
I20260927 07:05:39.573489 2766 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:39.573657 2766 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:39.573844 2766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:39.576666 2766 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d86eece16244288f01e221755902c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32953 } }
I20260927 07:05:39.577199 2766 leader_election.cc:302] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6d86eece16244288f01e221755902c4; no voters:
I20260927 07:05:39.577854 2766 leader_election.cc:288] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:39.577984 2776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:39.579071 2776 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [term 1 LEADER]: Becoming Leader. State: Replica: b6d86eece16244288f01e221755902c4, State: Running, Role: LEADER
I20260927 07:05:39.579748 2776 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d86eece16244288f01e221755902c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32953 } }
I20260927 07:05:39.580277 2766 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:39.586380 2777 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6d86eece16244288f01e221755902c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d86eece16244288f01e221755902c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32953 } } }
I20260927 07:05:39.586839 2777 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:39.587515 2778 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6d86eece16244288f01e221755902c4. Latest consensus state: current_term: 1 leader_uuid: "b6d86eece16244288f01e221755902c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d86eece16244288f01e221755902c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32953 } } }
I20260927 07:05:39.587813 2778 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:39.589138 2769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:39.572748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:47209 (local address 127.0.0.1:32953)
0927 07:05:39.572921 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:39.572927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:39.573367 (+ 440us) server_negotiation.cc:407] Connection header received
0927 07:05:39.573578 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:39.573584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:39.573673 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:39.573778 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:39.575214 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:39.576348 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:39.577717 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:39.578327 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:39.578474 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:39.581331 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:39.581336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:39.581340 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:39.581373 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:39.584559 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:39.585489 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:39.585495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:39.585498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:39.585675 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:39.586244 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:39.586249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:39.586252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:39.586809 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:39.587117 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:39.588364 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0927 07:05:39.588603 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:05:39.596256 2784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:39.601744 2784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:39.611563 2784 catalog_manager.cc:1347] Generated new cluster ID: ed872543922c438b9a3da855d2cc8efd
I20260927 07:05:39.611696 2784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:39.636861 2784 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:39.638345 2784 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:39.648029 2784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6d86eece16244288f01e221755902c4: Generated new TSK 0
I20260927 07:05:39.648864 2784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:39.658282 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2699
I20260927 07:05:39.660369 2761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:39.752305 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 2699
W20260927 07:05:39.767009 2684 connection.cc:535] client connection to 127.0.0.1:32953 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32953): Connection reset by peer (error 104)
2026-09-27T07:05:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1928 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:39 dist-test-slave-p6hz krb5kdc[2799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:39 dist-test-slave-p6hz krb5kdc[2799](info): set up 2 sockets
Sep 27 07:05:39 dist-test-slave-p6hz krb5kdc[2799](info): commencing operation
krb5kdc: starting...
W20260927 07:05:40.771680 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:40 dist-test-slave-p6hz krb5kdc[2799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:41.022238 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45289
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:41.350625 2812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:41.350926 2812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:41.351055 2812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:41.358471 2812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:41.358604 2812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:41.358654 2812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:41.358736 2812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:41.358788 2812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:41.369892 2812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45289
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:41.371359 2812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:41.372686 2812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:41.385169 2818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:41.385200 2820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:41.385236 2817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:41.386025 2812 server_base.cc:1034] running on GCE node
I20260927 07:05:41.387363 2812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:41.388998 2812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:41.390252 2812 hybrid_clock.cc:648] HybridClock initialized: now 1790492741390198 us; error 49 us; skew 500 ppm
I20260927 07:05:41.390661 2812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:41 dist-test-slave-p6hz krb5kdc[2799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492741, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:41.394029 2812 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:41.395668 2812 webserver.cc:460] Webserver started at http://127.0.0.1:46299/ using document root <none> and password file <none>
I20260927 07:05:41.396358 2812 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:41.396477 2812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:41.396765 2812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:41.399875 2812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "7a8391c99bc349488c05f1f156fa5153"
format_stamp: "Formatted at 2026-09-27 07:05:41 on dist-test-slave-p6hz"
server_key: "43cc388bcdb31a070cb8195e6128ba34"
server_key_iv: "dc8114d6fe88a4fa1bd92ef0a505ff50"
server_key_version: "encryptionkey@0"
I20260927 07:05:41.400676 2812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "7a8391c99bc349488c05f1f156fa5153"
format_stamp: "Formatted at 2026-09-27 07:05:41 on dist-test-slave-p6hz"
server_key: "43cc388bcdb31a070cb8195e6128ba34"
server_key_iv: "dc8114d6fe88a4fa1bd92ef0a505ff50"
server_key_version: "encryptionkey@0"
I20260927 07:05:41.406160 2812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:05:41.409974 2827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:41.411520 2812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260927 07:05:41.411689 2812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "7a8391c99bc349488c05f1f156fa5153"
format_stamp: "Formatted at 2026-09-27 07:05:41 on dist-test-slave-p6hz"
server_key: "43cc388bcdb31a070cb8195e6128ba34"
server_key_iv: "dc8114d6fe88a4fa1bd92ef0a505ff50"
server_key_version: "encryptionkey@0"
I20260927 07:05:41.411886 2812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:41.425812 2812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:41.430012 2812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:41.430285 2812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:41.447880 2812 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45289
I20260927 07:05:41.447955 2878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45289 every 8 connection(s)
I20260927 07:05:41.449224 2812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:41.452929 2879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:41.456606 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 2812
I20260927 07:05:41.456776 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:41.457192 13799 external_mini_cluster.cc:1428] Setting key 69e612a1e799302d269233744b02901e
I20260927 07:05:41.465751 2879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153: Bootstrap starting.
I20260927 07:05:41.469728 2879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:41.470947 2879 log.cc:824] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:41.471318 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:41.459125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:60015 (local address 127.0.0.1:45289)
0927 07:05:41.459923 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:41.459936 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:41.459981 (+ 45us) server_negotiation.cc:407] Connection header received
0927 07:05:41.461703 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:41.461730 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:41.462234 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:41.462606 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:41.463265 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:41.464644 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:41.466484 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:41.467274 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:41.467426 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:41.468734 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:41.468742 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:41.468758 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:41.469300 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:41.469379 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:41.469405 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:41.469625 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:41.469952 (+ 327us) server_negotiation.cc:299] Negotiation successful
0927 07:05:41.470178 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":344,"threads_started":1}
I20260927 07:05:41.475008 2879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153: No bootstrap required, opened a new log
I20260927 07:05:41.475104 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:41.483716 2879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8391c99bc349488c05f1f156fa5153" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45289 } }
I20260927 07:05:41.484200 2879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:41.484355 2879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a8391c99bc349488c05f1f156fa5153, State: Initialized, Role: FOLLOWER
I20260927 07:05:41.485001 2879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8391c99bc349488c05f1f156fa5153" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45289 } }
I20260927 07:05:41.485359 2879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:41.485586 2879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:41.485798 2879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:41.485944 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:41.479295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:46135 (local address 127.0.0.1:45289)
0927 07:05:41.479474 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:41.479480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:41.479756 (+ 276us) server_negotiation.cc:407] Connection header received
0927 07:05:41.479913 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:41.479919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:41.480003 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:41.480124 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:41.480556 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:41.481728 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:41.483012 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:41.483648 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:41.483778 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:41.484909 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:41.484916 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:41.484920 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:41.484943 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:41.484997 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:41.485018 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:41.485169 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:41.485554 (+ 385us) server_negotiation.cc:299] Negotiation successful
0927 07:05:41.485612 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 07:05:41.488575 2879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8391c99bc349488c05f1f156fa5153" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45289 } }
I20260927 07:05:41.489112 2879 leader_election.cc:302] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a8391c99bc349488c05f1f156fa5153; no voters:
I20260927 07:05:41.489660 2879 leader_election.cc:288] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:41.489792 2889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:41.490437 2889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [term 1 LEADER]: Becoming Leader. State: Replica: 7a8391c99bc349488c05f1f156fa5153, State: Running, Role: LEADER
I20260927 07:05:41.491017 2889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8391c99bc349488c05f1f156fa5153" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45289 } }
I20260927 07:05:41.491714 2879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:41.493515 2890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a8391c99bc349488c05f1f156fa5153" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8391c99bc349488c05f1f156fa5153" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45289 } } }
I20260927 07:05:41.493923 2890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:41.493922 2891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a8391c99bc349488c05f1f156fa5153. Latest consensus state: current_term: 1 leader_uuid: "7a8391c99bc349488c05f1f156fa5153" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8391c99bc349488c05f1f156fa5153" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45289 } } }
I20260927 07:05:41.494234 2891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:41.499933 2894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:41.505112 2894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:41.516033 2894 catalog_manager.cc:1347] Generated new cluster ID: a1beef6cafd742d69412b4a96ad8645e
I20260927 07:05:41.516201 2894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:41.536950 2894 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:41.538380 2894 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:41.550172 2894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a8391c99bc349488c05f1f156fa5153: Generated new TSK 0
I20260927 07:05:41.551101 2894 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:41.557701 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2812
W20260927 07:05:41.558900 2874 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.90:60015
W20260927 07:05:41.559777 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2812: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:41.559964 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 2812
W20260927 07:05:41.574230 2797 connection.cc:535] client connection to 127.0.0.1:45289 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45289): Connection reset by peer (error 104)
2026-09-27T07:05:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1806 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:41 dist-test-slave-p6hz krb5kdc[2910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:41 dist-test-slave-p6hz krb5kdc[2910](info): set up 2 sockets
Sep 27 07:05:41 dist-test-slave-p6hz krb5kdc[2910](info): commencing operation
krb5kdc: starting...
W20260927 07:05:42.587540 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:42 dist-test-slave-p6hz krb5kdc[2910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 07:05:42.836170 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:43.165697 2923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:43.166016 2923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:43.166157 2923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:43.174903 2923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:43.175050 2923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:43.175196 2923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:43.175287 2923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:43.175351 2923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:43.185637 2923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41745
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:43.186944 2923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:43.188310 2923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:43.200083 2928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:43.199986 2931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:43.200028 2929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:43.200917 2923 server_base.cc:1034] running on GCE node
I20260927 07:05:43.201784 2923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:43.203311 2923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:43.204533 2923 hybrid_clock.cc:648] HybridClock initialized: now 1790492743204480 us; error 56 us; skew 500 ppm
I20260927 07:05:43.204897 2923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:43 dist-test-slave-p6hz krb5kdc[2910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492743, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:43.212477 2923 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 07:05:43.214224 2923 webserver.cc:460] Webserver started at http://127.0.0.1:42837/ using document root <none> and password file <none>
I20260927 07:05:43.214970 2923 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:43.215101 2923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:43.215440 2923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:43.218514 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "0edf7fed92fa4c74aec0780003bfb325"
format_stamp: "Formatted at 2026-09-27 07:05:43 on dist-test-slave-p6hz"
server_key: "2d767ecd7a1448fbc68a309829956e49"
server_key_iv: "603e287a650edbd23f965c9c1c784473"
server_key_version: "encryptionkey@0"
I20260927 07:05:43.219333 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "0edf7fed92fa4c74aec0780003bfb325"
format_stamp: "Formatted at 2026-09-27 07:05:43 on dist-test-slave-p6hz"
server_key: "2d767ecd7a1448fbc68a309829956e49"
server_key_iv: "603e287a650edbd23f965c9c1c784473"
server_key_version: "encryptionkey@0"
I20260927 07:05:43.224577 2923 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:05:43.228056 2938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:43.229422 2923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:43.229597 2923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "0edf7fed92fa4c74aec0780003bfb325"
format_stamp: "Formatted at 2026-09-27 07:05:43 on dist-test-slave-p6hz"
server_key: "2d767ecd7a1448fbc68a309829956e49"
server_key_iv: "603e287a650edbd23f965c9c1c784473"
server_key_version: "encryptionkey@0"
I20260927 07:05:43.229795 2923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:43.235441 2923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:43.239554 2923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:43.239830 2923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:43.257246 2923 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41745
I20260927 07:05:43.257293 2989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41745 every 8 connection(s)
I20260927 07:05:43.258687 2923 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:43.259682 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 2923
I20260927 07:05:43.259859 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:43.260365 13799 external_mini_cluster.cc:1428] Setting key 075c54e7503e62d1eca01ab203bf4463
I20260927 07:05:43.263309 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:43.268194 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:43.262175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:54365 (local address 127.0.0.1:41745)
0927 07:05:43.262987 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:43.263001 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:43.263045 (+ 44us) server_negotiation.cc:407] Connection header received
0927 07:05:43.264846 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:43.264870 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:43.265197 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:43.265570 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:43.266533 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:43.266542 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:43.266555 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:43.266573 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:43.266639 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:43.266663 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:43.266868 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:43.267017 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 07:05:43.267252 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":319,"threads_started":1}
I20260927 07:05:43.272051 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:43.274920 2990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325: Bootstrap starting.
I20260927 07:05:43.278879 2990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:43.280102 2990 log.cc:824] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:43.281360 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:43.276299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.90:41447 (local address 127.0.0.1:41745)
0927 07:05:43.276455 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:43.276460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:43.279489 (+ 3029us) server_negotiation.cc:407] Connection header received
0927 07:05:43.279667 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:43.279670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:43.279721 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:43.279805 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:43.280725 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:43.280733 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:43.280738 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:43.280756 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:43.280799 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:43.280814 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:43.280916 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:43.281009 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 07:05:43.281079 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 07:05:43.283207 2990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325: No bootstrap required, opened a new log
I20260927 07:05:43.290540 2990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0edf7fed92fa4c74aec0780003bfb325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } }
I20260927 07:05:43.290896 2990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:43.290988 2990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0edf7fed92fa4c74aec0780003bfb325, State: Initialized, Role: FOLLOWER
I20260927 07:05:43.291567 2990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0edf7fed92fa4c74aec0780003bfb325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } }
I20260927 07:05:43.291774 2990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:43.291906 2990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:43.292092 2990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:43.294186 2990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0edf7fed92fa4c74aec0780003bfb325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } }
I20260927 07:05:43.294658 2990 leader_election.cc:302] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0edf7fed92fa4c74aec0780003bfb325; no voters:
I20260927 07:05:43.295248 2990 leader_election.cc:288] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:43.295377 3000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:43.295975 3000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [term 1 LEADER]: Becoming Leader. State: Replica: 0edf7fed92fa4c74aec0780003bfb325, State: Running, Role: LEADER
I20260927 07:05:43.296505 3000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0edf7fed92fa4c74aec0780003bfb325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } }
I20260927 07:05:43.297261 2990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:43.299185 3001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0edf7fed92fa4c74aec0780003bfb325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0edf7fed92fa4c74aec0780003bfb325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } } }
I20260927 07:05:43.299690 3001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:43.299336 3002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0edf7fed92fa4c74aec0780003bfb325. Latest consensus state: current_term: 1 leader_uuid: "0edf7fed92fa4c74aec0780003bfb325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0edf7fed92fa4c74aec0780003bfb325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41745 } } }
I20260927 07:05:43.300261 3002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:43.300770 3009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:43.305579 3009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:43.314200 3009 catalog_manager.cc:1347] Generated new cluster ID: 17ef3dba20124db3a276e9de06fbff29
I20260927 07:05:43.314327 3009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:43.330246 3009 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:43.330783 3009 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:43.346271 3009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0edf7fed92fa4c74aec0780003bfb325: Generated new TSK 0
I20260927 07:05:43.347440 3009 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:43.357735 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2923
W20260927 07:05:43.359323 2985 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.90:54365
W20260927 07:05:43.360131 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 2923: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:43.360277 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 2923
2026-09-27T07:05:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1799 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:43 dist-test-slave-p6hz krb5kdc[3021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:43 dist-test-slave-p6hz krb5kdc[3021](info): set up 2 sockets
Sep 27 07:05:43 dist-test-slave-p6hz krb5kdc[3021](info): commencing operation
krb5kdc: starting...
W20260927 07:05:44.379597 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:44 dist-test-slave-p6hz krb5kdc[3021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:44.638669 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38603
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:44401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:38603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:44.982179 3034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:44.982519 3034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:44.982661 3034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:44.990067 3034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:44.990211 3034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:44.990286 3034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:44.990365 3034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:44.990424 3034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:45.001021 3034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:44401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:38603
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:38603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:45.002398 3034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:45.003782 3034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:45.015520 3039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:45.015620 3042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:45.015579 3040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:45.016726 3034 server_base.cc:1034] running on GCE node
I20260927 07:05:45.017511 3034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:45.018921 3034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:45.020128 3034 hybrid_clock.cc:648] HybridClock initialized: now 1790492745020065 us; error 47 us; skew 500 ppm
I20260927 07:05:45.020485 3034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:45 dist-test-slave-p6hz krb5kdc[3021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492745, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:45.023595 3034 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:45.025039 3034 webserver.cc:460] Webserver started at http://127.13.121.254:36837/ using document root <none> and password file <none>
I20260927 07:05:45.025717 3034 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:45.025830 3034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:45.026104 3034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:45.029072 3034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72"
format_stamp: "Formatted at 2026-09-27 07:05:45 on dist-test-slave-p6hz"
server_key: "e016b5451b55b3ce085e0de473318dfe"
server_key_iv: "c1562737e0d6e103c79dc8f00b5a2250"
server_key_version: "encryptionkey@0"
I20260927 07:05:45.029806 3034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72"
format_stamp: "Formatted at 2026-09-27 07:05:45 on dist-test-slave-p6hz"
server_key: "e016b5451b55b3ce085e0de473318dfe"
server_key_iv: "c1562737e0d6e103c79dc8f00b5a2250"
server_key_version: "encryptionkey@0"
I20260927 07:05:45.034874 3034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260927 07:05:45.038357 3049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:45.039884 3034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:45.040093 3034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72"
format_stamp: "Formatted at 2026-09-27 07:05:45 on dist-test-slave-p6hz"
server_key: "e016b5451b55b3ce085e0de473318dfe"
server_key_iv: "c1562737e0d6e103c79dc8f00b5a2250"
server_key_version: "encryptionkey@0"
I20260927 07:05:45.040292 3034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:45.067842 3034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:45.072558 3034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:45.072865 3034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:45.090929 3034 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:38603
I20260927 07:05:45.090934 3100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:38603 every 8 connection(s)
I20260927 07:05:45.092338 3034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:45.094579 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 3034
I20260927 07:05:45.094732 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:45.095188 13799 external_mini_cluster.cc:1428] Setting key ca3c9f6f317f99e4227427ce591ba7d4
I20260927 07:05:45.096439 3101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 07:05:45 dist-test-slave-p6hz krb5kdc[3021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:45.110102 3101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72: Bootstrap starting.
I20260927 07:05:45.114243 3101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:45.115516 3101 log.cc:824] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:45.119179 3101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72: No bootstrap required, opened a new log
I20260927 07:05:45.119540 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:45.096835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47449 (local address 127.13.121.254:38603)
0927 07:05:45.097588 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:45.097604 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:45.097655 (+ 51us) server_negotiation.cc:407] Connection header received
0927 07:05:45.099609 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:45.099637 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:45.100161 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:45.100619 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:45.101924 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:45.103988 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:45.105308 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:45.106323 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:45.106509 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:45.110742 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:45.110754 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:45.110775 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:45.110828 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:45.114743 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:45.115691 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:45.115699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:45.115708 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:45.115929 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:45.116465 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:45.116469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:45.116471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:45.117108 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:45.117426 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:45.117827 (+ 401us) server_negotiation.cc:299] Negotiation successful
0927 07:05:45.118178 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":290,"threads_started":1}
I20260927 07:05:45.123587 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:45.128918 3101 raft_consensus.cc:348] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38603 } }
I20260927 07:05:45.129359 3101 raft_consensus.cc:374] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:45.129462 3101 raft_consensus.cc:729] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84346fdb3b84c75b4f2a6fc3e0b3c72, State: Initialized, Role: FOLLOWER
I20260927 07:05:45.130100 3101 consensus_queue.cc:260] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38603 } }
I20260927 07:05:45.130313 3101 raft_consensus.cc:388] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:45.130446 3101 raft_consensus.cc:482] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:45.130595 3101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:45.133421 3101 raft_consensus.cc:504] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38603 } }
I20260927 07:05:45.134001 3101 leader_election.cc:302] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b84346fdb3b84c75b4f2a6fc3e0b3c72; no voters:
I20260927 07:05:45.134732 3101 leader_election.cc:288] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:45.134872 3111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:45.136915 3101 sys_catalog.cc:564] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:45.138118 3111 raft_consensus.cc:686] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [term 1 LEADER]: Becoming Leader. State: Replica: b84346fdb3b84c75b4f2a6fc3e0b3c72, State: Running, Role: LEADER
I20260927 07:05:45.138831 3111 consensus_queue.cc:237] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38603 } }
I20260927 07:05:45.142177 3113 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b84346fdb3b84c75b4f2a6fc3e0b3c72. Latest consensus state: current_term: 1 leader_uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38603 } } }
I20260927 07:05:45.142583 3113 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:45.143164 3112 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84346fdb3b84c75b4f2a6fc3e0b3c72" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 38603 } } }
I20260927 07:05:45.143482 3112 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:45.147559 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:45.128776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33589 (local address 127.13.121.254:38603)
0927 07:05:45.128965 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:45.128972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:45.129430 (+ 458us) server_negotiation.cc:407] Connection header received
0927 07:05:45.129615 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:45.129620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:45.129698 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:45.129799 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:45.131438 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:45.132578 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:45.133807 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:45.134464 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:45.134607 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:45.137939 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:45.137947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:45.137953 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:45.138005 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:45.142403 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:45.143668 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:45.143675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:45.143679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:45.143898 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:45.144518 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:45.144524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:45.144527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:45.144937 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:45.146475 (+ 1538us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:45.146684 (+ 209us) server_negotiation.cc:299] Negotiation successful
0927 07:05:45.146898 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 07:05:45.151921 3119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:45.158234 3119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:45.168022 3119 catalog_manager.cc:1347] Generated new cluster ID: f1c5a8d9c8e24d7f8d25141291515da7
I20260927 07:05:45.168145 3119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:45.191437 3119 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:45.192858 3119 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:45.201202 3119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b84346fdb3b84c75b4f2a6fc3e0b3c72: Generated new TSK 0
I20260927 07:05:45.202154 3119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:45.219529 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3034
I20260927 07:05:45.220917 3096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:45.314709 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 3034
W20260927 07:05:45.329058 3019 connection.cc:535] client connection to 127.13.121.254:38603 recv error: Network error: failed to read from TLS socket (remote: 127.13.121.254:38603): Connection reset by peer (error 104)
2026-09-27T07:05:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1954 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:45 dist-test-slave-p6hz krb5kdc[3134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:45 dist-test-slave-p6hz krb5kdc[3134](info): set up 2 sockets
Sep 27 07:05:45 dist-test-slave-p6hz krb5kdc[3134](info): commencing operation
krb5kdc: starting...
W20260927 07:05:46.331465 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:46 dist-test-slave-p6hz krb5kdc[3134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:46.583277 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:33711
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:33711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:46.917807 3147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:46.918176 3147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:46.918308 3147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:46.926385 3147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:46.926667 3147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:46.926764 3147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:46.926851 3147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:46.926910 3147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:46.937408 3147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:33711
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:33711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:46.938866 3147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:46.940254 3147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:46.952454 3152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:46.952662 3153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:46.952662 3155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:46.953768 3147 server_base.cc:1034] running on GCE node
I20260927 07:05:46.954638 3147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:46.956208 3147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:46.957437 3147 hybrid_clock.cc:648] HybridClock initialized: now 1790492746957391 us; error 37 us; skew 500 ppm
I20260927 07:05:46.957855 3147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:46 dist-test-slave-p6hz krb5kdc[3134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492746, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:46.961158 3147 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:46.962762 3147 webserver.cc:460] Webserver started at http://127.13.121.254:38931/ using document root <none> and password file <none>
I20260927 07:05:46.963614 3147 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:46.963743 3147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:46.964038 3147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:46.967088 3147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "545ed043ebcc4e06bc6f87bb84bc8be8"
format_stamp: "Formatted at 2026-09-27 07:05:46 on dist-test-slave-p6hz"
server_key: "7d10ce0c9990bc082efdd92ae4dfce47"
server_key_iv: "c63bfd227378a4f42eefe419d39a1be2"
server_key_version: "encryptionkey@0"
I20260927 07:05:46.967893 3147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "545ed043ebcc4e06bc6f87bb84bc8be8"
format_stamp: "Formatted at 2026-09-27 07:05:46 on dist-test-slave-p6hz"
server_key: "7d10ce0c9990bc082efdd92ae4dfce47"
server_key_iv: "c63bfd227378a4f42eefe419d39a1be2"
server_key_version: "encryptionkey@0"
I20260927 07:05:46.973131 3147 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:05:46.976802 3162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:46.978497 3147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260927 07:05:46.978721 3147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "545ed043ebcc4e06bc6f87bb84bc8be8"
format_stamp: "Formatted at 2026-09-27 07:05:46 on dist-test-slave-p6hz"
server_key: "7d10ce0c9990bc082efdd92ae4dfce47"
server_key_iv: "c63bfd227378a4f42eefe419d39a1be2"
server_key_version: "encryptionkey@0"
I20260927 07:05:46.979074 3147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:47.000212 3147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:47.004654 3147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:47.004942 3147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:47.023074 3147 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:33711
I20260927 07:05:47.023108 3213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:33711 every 8 connection(s)
I20260927 07:05:47.024506 3147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:47.027869 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 3147
I20260927 07:05:47.028074 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:47.028566 13799 external_mini_cluster.cc:1428] Setting key 573ae426b3ba962204d7f300cef5e46d
I20260927 07:05:47.028808 3214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:47.042845 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:47.030350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60241 (local address 127.13.121.254:33711)
0927 07:05:47.031129 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:47.031173 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:47.031221 (+ 48us) server_negotiation.cc:407] Connection header received
0927 07:05:47.033474 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:47.033501 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:47.033964 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:47.034338 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:47.034961 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:47.036395 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:47.038181 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:47.038889 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:47.039045 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:47.040373 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:47.040381 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:47.040397 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:47.040997 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:47.041100 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:47.041128 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:47.041256 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:47.041562 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 07:05:47.041778 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":358,"threads_started":1}
I20260927 07:05:47.042840 3214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8: Bootstrap starting.
I20260927 07:05:47.046139 3214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:47.046655 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:47.047334 3214 log.cc:824] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:47.050738 3214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8: No bootstrap required, opened a new log
I20260927 07:05:47.058270 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:47.051532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56215 (local address 127.13.121.254:33711)
0927 07:05:47.051704 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:47.051711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:47.052111 (+ 400us) server_negotiation.cc:407] Connection header received
0927 07:05:47.052283 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:47.052288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:47.052364 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:47.052481 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:47.052984 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:47.054109 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:47.055532 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:47.056253 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:47.056383 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:47.057499 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:47.057509 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:47.057514 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:47.057538 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:47.057607 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:47.057629 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:47.057737 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:47.057857 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 07:05:47.057913 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 07:05:47.060581 3214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545ed043ebcc4e06bc6f87bb84bc8be8" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33711 } }
I20260927 07:05:47.060976 3214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:47.061057 3214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545ed043ebcc4e06bc6f87bb84bc8be8, State: Initialized, Role: FOLLOWER
I20260927 07:05:47.061579 3214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545ed043ebcc4e06bc6f87bb84bc8be8" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33711 } }
I20260927 07:05:47.061784 3214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:47.061877 3214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:47.062004 3214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:47.064630 3214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545ed043ebcc4e06bc6f87bb84bc8be8" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33711 } }
I20260927 07:05:47.065138 3214 leader_election.cc:302] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545ed043ebcc4e06bc6f87bb84bc8be8; no voters:
I20260927 07:05:47.065685 3214 leader_election.cc:288] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:47.065821 3224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:47.066450 3224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [term 1 LEADER]: Becoming Leader. State: Replica: 545ed043ebcc4e06bc6f87bb84bc8be8, State: Running, Role: LEADER
I20260927 07:05:47.066955 3224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545ed043ebcc4e06bc6f87bb84bc8be8" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33711 } }
I20260927 07:05:47.067757 3214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:47.069802 3225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545ed043ebcc4e06bc6f87bb84bc8be8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545ed043ebcc4e06bc6f87bb84bc8be8" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33711 } } }
I20260927 07:05:47.069780 3226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545ed043ebcc4e06bc6f87bb84bc8be8. Latest consensus state: current_term: 1 leader_uuid: "545ed043ebcc4e06bc6f87bb84bc8be8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545ed043ebcc4e06bc6f87bb84bc8be8" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 33711 } } }
I20260927 07:05:47.070407 3226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:47.070425 3225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:47.076436 3229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:47.082089 3229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:47.092561 3229 catalog_manager.cc:1347] Generated new cluster ID: 9af020bfc928433c9599e38d9d615d00
I20260927 07:05:47.092689 3229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:47.105695 3229 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:47.107043 3229 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:47.122150 3229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545ed043ebcc4e06bc6f87bb84bc8be8: Generated new TSK 0
I20260927 07:05:47.123288 3229 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:47.140623 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3147
W20260927 07:05:47.142016 3209 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60241
W20260927 07:05:47.142820 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3147: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:47.142977 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 3147
W20260927 07:05:47.158183 3132 connection.cc:535] client connection to 127.13.121.254:33711 recv error: Network error: failed to read from TLS socket (remote: 127.13.121.254:33711): Connection reset by peer (error 104)
2026-09-27T07:05:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1829 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:47 dist-test-slave-p6hz krb5kdc[3245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:47 dist-test-slave-p6hz krb5kdc[3245](info): set up 2 sockets
Sep 27 07:05:47 dist-test-slave-p6hz krb5kdc[3245](info): commencing operation
krb5kdc: starting...
W20260927 07:05:48.171599 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:48 dist-test-slave-p6hz krb5kdc[3245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:48.424155 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41103
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:48.752887 3258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:48.753201 3258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:48.753321 3258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:48.760843 3258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:48.760977 3258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:48.761052 3258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:48.761145 3258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:48.761197 3258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:48.771339 3258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:48.772651 3258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:48.773921 3258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:48.785386 3263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:48.785485 3266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:48.785373 3264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:48.786231 3258 server_base.cc:1034] running on GCE node
I20260927 07:05:48.787051 3258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:48.788610 3258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:48.789829 3258 hybrid_clock.cc:648] HybridClock initialized: now 1790492748789771 us; error 53 us; skew 500 ppm
I20260927 07:05:48.790187 3258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:48 dist-test-slave-p6hz krb5kdc[3245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492748, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:48.799089 3258 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:48.800730 3258 webserver.cc:460] Webserver started at http://127.13.121.254:39419/ using document root <none> and password file <none>
I20260927 07:05:48.801404 3258 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:48.801507 3258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:48.801762 3258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:48.804832 3258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "0aaec24bfe09474a8f512199432545ec"
format_stamp: "Formatted at 2026-09-27 07:05:48 on dist-test-slave-p6hz"
server_key: "62f248543c988ea6d7b7e490b9e9e6bf"
server_key_iv: "225d716a14ccf746d6de522505fc1b9f"
server_key_version: "encryptionkey@0"
I20260927 07:05:48.805608 3258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "0aaec24bfe09474a8f512199432545ec"
format_stamp: "Formatted at 2026-09-27 07:05:48 on dist-test-slave-p6hz"
server_key: "62f248543c988ea6d7b7e490b9e9e6bf"
server_key_iv: "225d716a14ccf746d6de522505fc1b9f"
server_key_version: "encryptionkey@0"
I20260927 07:05:48.810954 3258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 07:05:48.814702 3273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:48.816277 3258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:05:48.816479 3258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "0aaec24bfe09474a8f512199432545ec"
format_stamp: "Formatted at 2026-09-27 07:05:48 on dist-test-slave-p6hz"
server_key: "62f248543c988ea6d7b7e490b9e9e6bf"
server_key_iv: "225d716a14ccf746d6de522505fc1b9f"
server_key_version: "encryptionkey@0"
I20260927 07:05:48.816691 3258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:48.822772 3258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:48.826934 3258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:48.827253 3258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:48.844786 3258 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:41103
I20260927 07:05:48.844817 3324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:41103 every 8 connection(s)
I20260927 07:05:48.846158 3258 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:48.848318 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 3258
I20260927 07:05:48.848508 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:48.849078 13799 external_mini_cluster.cc:1428] Setting key 48d8627e16b2a48cfd9dceba93c3cc95
I20260927 07:05:48.850608 3325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:48.856753 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:48.850845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35467 (local address 127.13.121.254:41103)
0927 07:05:48.851631 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:48.851643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:48.851684 (+ 41us) server_negotiation.cc:407] Connection header received
0927 07:05:48.853244 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:48.853266 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:48.853636 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:48.853987 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:48.855022 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:48.855028 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:48.855041 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:48.855061 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:48.855183 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:48.855218 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:48.855479 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:48.855623 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 07:05:48.855803 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":353,"threads_started":1}
I20260927 07:05:48.860388 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:48.862864 3325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec: Bootstrap starting.
I20260927 07:05:48.866626 3325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:48.867024 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:48.864644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49281 (local address 127.13.121.254:41103)
0927 07:05:48.864791 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:48.864795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:48.865093 (+ 298us) server_negotiation.cc:407] Connection header received
0927 07:05:48.865237 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:48.865241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:48.865293 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:48.865374 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:48.866304 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:48.866308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:48.866310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:48.866325 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:48.866362 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:48.866377 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:48.866473 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:48.866641 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 07:05:48.866691 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 07:05:48.867853 3325 log.cc:824] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:48.870642 3325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec: No bootstrap required, opened a new log
I20260927 07:05:48.878290 3325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aaec24bfe09474a8f512199432545ec" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41103 } }
I20260927 07:05:48.878778 3325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:48.878886 3325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aaec24bfe09474a8f512199432545ec, State: Initialized, Role: FOLLOWER
I20260927 07:05:48.879539 3325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aaec24bfe09474a8f512199432545ec" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41103 } }
I20260927 07:05:48.879755 3325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:48.879853 3325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:48.879985 3325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:48.882355 3325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aaec24bfe09474a8f512199432545ec" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41103 } }
I20260927 07:05:48.882990 3325 leader_election.cc:302] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0aaec24bfe09474a8f512199432545ec; no voters:
I20260927 07:05:48.883728 3325 leader_election.cc:288] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:48.883845 3335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:48.884398 3335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [term 1 LEADER]: Becoming Leader. State: Replica: 0aaec24bfe09474a8f512199432545ec, State: Running, Role: LEADER
I20260927 07:05:48.885018 3335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aaec24bfe09474a8f512199432545ec" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41103 } }
I20260927 07:05:48.886019 3325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:48.887828 3337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0aaec24bfe09474a8f512199432545ec. Latest consensus state: current_term: 1 leader_uuid: "0aaec24bfe09474a8f512199432545ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aaec24bfe09474a8f512199432545ec" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41103 } } }
I20260927 07:05:48.888255 3337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:48.889410 3336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0aaec24bfe09474a8f512199432545ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aaec24bfe09474a8f512199432545ec" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41103 } } }
I20260927 07:05:48.889850 3336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:48.891368 3344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:48.896718 3344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:48.905653 3344 catalog_manager.cc:1347] Generated new cluster ID: b596e8aa3641425f9b5491bf69d1d8bf
I20260927 07:05:48.905790 3344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:48.919490 3344 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:48.919957 3344 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:48.931496 3344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0aaec24bfe09474a8f512199432545ec: Generated new TSK 0
I20260927 07:05:48.932420 3344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:48.941324 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3258
W20260927 07:05:48.942370 3320 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35467
W20260927 07:05:48.943023 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3258: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:48.943167 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 3258
2026-09-27T07:05:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1799 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:49 dist-test-slave-p6hz krb5kdc[3356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:49 dist-test-slave-p6hz krb5kdc[3356](info): set up 2 sockets
Sep 27 07:05:49 dist-test-slave-p6hz krb5kdc[3356](info): commencing operation
krb5kdc: starting...
W20260927 07:05:49.963496 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:50 dist-test-slave-p6hz krb5kdc[3356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:50.218645 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41779
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:35561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:50.552037 3369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:50.552332 3369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:50.552464 3369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:50.560154 3369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:50.560288 3369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:50.560352 3369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:50.560429 3369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:50.560487 3369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:50.570520 3369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:35561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:50.571853 3369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:50.573127 3369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:50.584667 3374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:50.584703 3377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:50.584743 3375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:50.585197 3369 server_base.cc:1034] running on GCE node
I20260927 07:05:50.586251 3369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:50.587819 3369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:50.589022 3369 hybrid_clock.cc:648] HybridClock initialized: now 1790492750588977 us; error 37 us; skew 500 ppm
I20260927 07:05:50.589345 3369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:50 dist-test-slave-p6hz krb5kdc[3356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492750, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:50.592412 3369 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:50.593961 3369 webserver.cc:460] Webserver started at http://127.13.121.254:38295/ using document root <none> and password file <none>
I20260927 07:05:50.594641 3369 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:50.594743 3369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:50.595005 3369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:50.598253 3369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "c63e6219ae9b4b34a50f8ce9962d8c91"
format_stamp: "Formatted at 2026-09-27 07:05:50 on dist-test-slave-p6hz"
server_key: "e63b76568eef8222db7ae094a17a3c64"
server_key_iv: "ada85e00457fe7f288265ff0b80a2c03"
server_key_version: "encryptionkey@0"
I20260927 07:05:50.599071 3369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "c63e6219ae9b4b34a50f8ce9962d8c91"
format_stamp: "Formatted at 2026-09-27 07:05:50 on dist-test-slave-p6hz"
server_key: "e63b76568eef8222db7ae094a17a3c64"
server_key_iv: "ada85e00457fe7f288265ff0b80a2c03"
server_key_version: "encryptionkey@0"
I20260927 07:05:50.604401 3369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260927 07:05:50.607928 3384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:50.609331 3369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:50.609493 3369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "c63e6219ae9b4b34a50f8ce9962d8c91"
format_stamp: "Formatted at 2026-09-27 07:05:50 on dist-test-slave-p6hz"
server_key: "e63b76568eef8222db7ae094a17a3c64"
server_key_iv: "ada85e00457fe7f288265ff0b80a2c03"
server_key_version: "encryptionkey@0"
I20260927 07:05:50.609685 3369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:50.635340 3369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:50.639633 3369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:50.639899 3369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:50.657282 3369 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:41779
I20260927 07:05:50.657303 3435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:41779 every 8 connection(s)
I20260927 07:05:50.658625 3369 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:50.662453 3436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:50.663185 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 3369
I20260927 07:05:50.663333 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:50.663743 13799 external_mini_cluster.cc:1428] Setting key cc115c7ca4c5a808f150cabe8b50164e
I20260927 07:05:50.676889 3436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91: Bootstrap starting.
Sep 27 07:05:50 dist-test-slave-p6hz krb5kdc[3356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:50.680147 3436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:50.681499 3436 log.cc:824] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:50.685038 3436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91: No bootstrap required, opened a new log
I20260927 07:05:50.688298 3439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:50.665602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56555 (local address 127.13.121.254:41779)
0927 07:05:50.666398 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:50.666414 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:50.666465 (+ 51us) server_negotiation.cc:407] Connection header received
0927 07:05:50.668920 (+ 2455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:50.668955 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:50.669551 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:50.670039 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:50.671409 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:50.673405 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:50.674740 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:50.675811 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:50.675988 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:50.679721 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:50.679732 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:50.679752 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:50.679804 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:50.683408 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:50.684162 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:50.684171 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:50.684180 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:50.684394 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:50.684801 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:50.684806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:50.684809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:50.685569 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:50.685862 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:50.686331 (+ 469us) server_negotiation.cc:299] Negotiation successful
0927 07:05:50.686795 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":327,"threads_started":1}
I20260927 07:05:50.692561 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:50.695775 3436 raft_consensus.cc:348] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c63e6219ae9b4b34a50f8ce9962d8c91" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41779 } }
I20260927 07:05:50.696276 3436 raft_consensus.cc:374] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:50.696393 3436 raft_consensus.cc:729] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c63e6219ae9b4b34a50f8ce9962d8c91, State: Initialized, Role: FOLLOWER
I20260927 07:05:50.697091 3436 consensus_queue.cc:260] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c63e6219ae9b4b34a50f8ce9962d8c91" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41779 } }
I20260927 07:05:50.697377 3436 raft_consensus.cc:388] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:50.697525 3436 raft_consensus.cc:482] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:50.697715 3436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:50.700212 3436 raft_consensus.cc:504] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c63e6219ae9b4b34a50f8ce9962d8c91" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41779 } }
I20260927 07:05:50.700830 3436 leader_election.cc:302] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c63e6219ae9b4b34a50f8ce9962d8c91; no voters:
I20260927 07:05:50.701499 3436 leader_election.cc:288] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:50.701629 3446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:50.702314 3446 raft_consensus.cc:686] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [term 1 LEADER]: Becoming Leader. State: Replica: c63e6219ae9b4b34a50f8ce9962d8c91, State: Running, Role: LEADER
I20260927 07:05:50.703020 3446 consensus_queue.cc:237] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c63e6219ae9b4b34a50f8ce9962d8c91" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41779 } }
I20260927 07:05:50.704985 3436 sys_catalog.cc:564] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:50.709420 3447 sys_catalog.cc:455] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c63e6219ae9b4b34a50f8ce9962d8c91. Latest consensus state: current_term: 1 leader_uuid: "c63e6219ae9b4b34a50f8ce9962d8c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c63e6219ae9b4b34a50f8ce9962d8c91" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41779 } } }
I20260927 07:05:50.709851 3447 sys_catalog.cc:458] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:50.708899 3448 sys_catalog.cc:455] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c63e6219ae9b4b34a50f8ce9962d8c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c63e6219ae9b4b34a50f8ce9962d8c91" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41779 } } }
I20260927 07:05:50.710551 3448 sys_catalog.cc:458] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:50.712016 3455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:50.714129 3439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:50.697623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40539 (local address 127.13.121.254:41779)
0927 07:05:50.697809 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:50.697814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:50.698507 (+ 693us) server_negotiation.cc:407] Connection header received
0927 07:05:50.698624 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:50.698638 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:50.698720 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:50.698827 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:50.700220 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:50.701247 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:50.703018 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:50.703786 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:50.703911 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:50.707003 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:50.707011 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:50.707017 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:50.707067 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:50.711059 (+ 3992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:50.711900 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:50.711906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:50.711908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:50.712039 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:50.712516 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:50.712522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:50.712526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:50.712986 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:50.713237 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:50.713550 (+ 313us) server_negotiation.cc:299] Negotiation successful
0927 07:05:50.713769 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 07:05:50.723860 3455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:50.733771 3455 catalog_manager.cc:1347] Generated new cluster ID: 20d241351d024c8d864053063cde85a4
I20260927 07:05:50.733901 3455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:50.758360 3455 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:50.760325 3455 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:50.780773 3455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c63e6219ae9b4b34a50f8ce9962d8c91: Generated new TSK 0
I20260927 07:05:50.781883 3455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:50.792755 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3369
I20260927 07:05:50.794209 3431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:50.888157 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 3369
2026-09-27T07:05:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1944 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:50 dist-test-slave-p6hz krb5kdc[3468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:50 dist-test-slave-p6hz krb5kdc[3468](info): set up 2 sockets
Sep 27 07:05:50 dist-test-slave-p6hz krb5kdc[3468](info): commencing operation
krb5kdc: starting...
W20260927 07:05:51.923663 13799 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:52 dist-test-slave-p6hz krb5kdc[3468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:52.191069 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:45421
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:45421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:52.528975 3481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:52.529299 3481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:52.529428 3481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:52.536798 3481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:52.536918 3481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:52.536998 3481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:52.537079 3481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:52.537142 3481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:52.547456 3481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:45421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:45421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:52.548782 3481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:52.550189 3481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:52.561772 3489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:52.561772 3487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:52.561888 3486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:52.562697 3481 server_base.cc:1034] running on GCE node
I20260927 07:05:52.563580 3481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:52.565011 3481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:52.566219 3481 hybrid_clock.cc:648] HybridClock initialized: now 1790492752566169 us; error 38 us; skew 500 ppm
I20260927 07:05:52.566562 3481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:52 dist-test-slave-p6hz krb5kdc[3468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492752, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:52.569623 3481 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:52.571184 3481 webserver.cc:460] Webserver started at http://127.13.121.254:46475/ using document root <none> and password file <none>
I20260927 07:05:52.571856 3481 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:52.571954 3481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:52.572207 3481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:52.575244 3481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "ffe3a3c0e4dd416594fe73d45d603d02"
format_stamp: "Formatted at 2026-09-27 07:05:52 on dist-test-slave-p6hz"
server_key: "cc6158bb0a79e6ba8df9dc19e907bc3f"
server_key_iv: "163e80326319a1e7a8a7e9e3201ee142"
server_key_version: "encryptionkey@0"
I20260927 07:05:52.576014 3481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "ffe3a3c0e4dd416594fe73d45d603d02"
format_stamp: "Formatted at 2026-09-27 07:05:52 on dist-test-slave-p6hz"
server_key: "cc6158bb0a79e6ba8df9dc19e907bc3f"
server_key_iv: "163e80326319a1e7a8a7e9e3201ee142"
server_key_version: "encryptionkey@0"
I20260927 07:05:52.581202 3481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 07:05:52.584939 3496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:52.586830 3481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 07:05:52.587046 3481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "ffe3a3c0e4dd416594fe73d45d603d02"
format_stamp: "Formatted at 2026-09-27 07:05:52 on dist-test-slave-p6hz"
server_key: "cc6158bb0a79e6ba8df9dc19e907bc3f"
server_key_iv: "163e80326319a1e7a8a7e9e3201ee142"
server_key_version: "encryptionkey@0"
I20260927 07:05:52.587337 3481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:52.606050 3481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:52.610507 3481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:52.610741 3481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:52.628491 3481 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:45421
I20260927 07:05:52.628491 3547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:45421 every 8 connection(s)
I20260927 07:05:52.629977 3481 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:52.633813 3548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:52.635421 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 3481
I20260927 07:05:52.635586 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:52.635989 13799 external_mini_cluster.cc:1428] Setting key e64b72912053cc90a7d3f633c32d9615
I20260927 07:05:52.647914 3548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02: Bootstrap starting.
I20260927 07:05:52.649986 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:52.637796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45775 (local address 127.13.121.254:45421)
0927 07:05:52.638636 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:52.638647 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:52.638690 (+ 43us) server_negotiation.cc:407] Connection header received
0927 07:05:52.640404 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:52.640434 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:52.640893 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:52.641249 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:52.641905 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:52.643411 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:52.645330 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:52.646119 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:52.646269 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:52.647559 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:52.647568 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:52.647584 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:52.648200 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:52.648289 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:52.648315 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:52.648412 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:52.648645 (+ 233us) server_negotiation.cc:299] Negotiation successful
0927 07:05:52.648913 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":369,"threads_started":1}
I20260927 07:05:52.651547 3548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:52.652694 3548 log.cc:824] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:52.653889 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:52.656075 3548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02: No bootstrap required, opened a new log
I20260927 07:05:52.665676 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:52.659045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37253 (local address 127.13.121.254:45421)
0927 07:05:52.659261 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:52.659268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:52.659656 (+ 388us) server_negotiation.cc:407] Connection header received
0927 07:05:52.659820 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:52.659825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:52.659913 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:52.660032 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:52.660539 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:52.661616 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:52.663009 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:52.663727 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:52.663856 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:52.664923 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:52.664930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:52.664935 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:52.664957 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:52.665004 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:52.665024 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:52.665115 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:52.665251 (+ 136us) server_negotiation.cc:299] Negotiation successful
0927 07:05:52.665348 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:05:52.665772 3548 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe3a3c0e4dd416594fe73d45d603d02" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45421 } }
I20260927 07:05:52.666189 3548 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:52.666306 3548 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe3a3c0e4dd416594fe73d45d603d02, State: Initialized, Role: FOLLOWER
I20260927 07:05:52.666932 3548 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe3a3c0e4dd416594fe73d45d603d02" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45421 } }
I20260927 07:05:52.667215 3548 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:52.667349 3548 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:52.667512 3548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:52.669687 3548 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe3a3c0e4dd416594fe73d45d603d02" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45421 } }
I20260927 07:05:52.670205 3548 leader_election.cc:302] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffe3a3c0e4dd416594fe73d45d603d02; no voters:
I20260927 07:05:52.670809 3548 leader_election.cc:288] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:52.670996 3558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:52.671733 3558 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [term 1 LEADER]: Becoming Leader. State: Replica: ffe3a3c0e4dd416594fe73d45d603d02, State: Running, Role: LEADER
I20260927 07:05:52.672266 3558 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe3a3c0e4dd416594fe73d45d603d02" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45421 } }
I20260927 07:05:52.672916 3548 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:52.674817 3560 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffe3a3c0e4dd416594fe73d45d603d02. Latest consensus state: current_term: 1 leader_uuid: "ffe3a3c0e4dd416594fe73d45d603d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe3a3c0e4dd416594fe73d45d603d02" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45421 } } }
I20260927 07:05:52.675334 3559 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffe3a3c0e4dd416594fe73d45d603d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffe3a3c0e4dd416594fe73d45d603d02" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 45421 } } }
I20260927 07:05:52.675560 3559 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:52.675887 3560 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:52.678437 3564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:52.683844 3564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:52.693041 3564 catalog_manager.cc:1347] Generated new cluster ID: 32cf465a38a24cd392a9cb754eefe6ec
I20260927 07:05:52.693199 3564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:52.721896 3564 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:52.723706 3564 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:52.732781 3564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffe3a3c0e4dd416594fe73d45d603d02: Generated new TSK 0
I20260927 07:05:52.733904 3564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:52.750648 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3481
W20260927 07:05:52.751796 3543 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45775
W20260927 07:05:52.752526 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3481: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:52.752660 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 3481
2026-09-27T07:05:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1864 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:52 dist-test-slave-p6hz krb5kdc[3579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:52 dist-test-slave-p6hz krb5kdc[3579](info): set up 2 sockets
Sep 27 07:05:52 dist-test-slave-p6hz krb5kdc[3579](info): commencing operation
krb5kdc: starting...
W20260927 07:05:53.779647 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:53 dist-test-slave-p6hz krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:54.030864 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41929
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:41381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:54.365710 3592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:54.366034 3592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:54.366173 3592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:54.373709 3592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:54.373842 3592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:54.373930 3592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:54.374014 3592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:54.374073 3592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:54.384419 3592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:41381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:54.385916 3592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:54.387445 3592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:54.399264 3597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:54.399248 3600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:54.399343 3598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:54.400755 3592 server_base.cc:1034] running on GCE node
I20260927 07:05:54.401677 3592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:54.403276 3592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:54.404510 3592 hybrid_clock.cc:648] HybridClock initialized: now 1790492754404460 us; error 49 us; skew 500 ppm
I20260927 07:05:54.404870 3592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:54 dist-test-slave-p6hz krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492754, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:54.412056 3592 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:54.413681 3592 webserver.cc:460] Webserver started at http://127.13.121.254:33213/ using document root <none> and password file <none>
I20260927 07:05:54.414361 3592 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:54.414490 3592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:54.414789 3592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:54.417866 3592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "017a075500004a83b35c558a60d6d62f"
format_stamp: "Formatted at 2026-09-27 07:05:54 on dist-test-slave-p6hz"
server_key: "d6cd48b8764b48dc3c3f5456b20dc4fb"
server_key_iv: "1f9dafcc9524628ba29a1bda23367e62"
server_key_version: "encryptionkey@0"
I20260927 07:05:54.418675 3592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "017a075500004a83b35c558a60d6d62f"
format_stamp: "Formatted at 2026-09-27 07:05:54 on dist-test-slave-p6hz"
server_key: "d6cd48b8764b48dc3c3f5456b20dc4fb"
server_key_iv: "1f9dafcc9524628ba29a1bda23367e62"
server_key_version: "encryptionkey@0"
I20260927 07:05:54.424180 3592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260927 07:05:54.427785 3607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:54.429234 3592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:54.429426 3592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "017a075500004a83b35c558a60d6d62f"
format_stamp: "Formatted at 2026-09-27 07:05:54 on dist-test-slave-p6hz"
server_key: "d6cd48b8764b48dc3c3f5456b20dc4fb"
server_key_iv: "1f9dafcc9524628ba29a1bda23367e62"
server_key_version: "encryptionkey@0"
I20260927 07:05:54.429633 3592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:54.435623 3592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:54.440639 3592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:54.440932 3592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:54.458388 3592 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:41929
I20260927 07:05:54.458427 3658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:41929 every 8 connection(s)
I20260927 07:05:54.459844 3592 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:54.463627 3659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:54.465420 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 3592
I20260927 07:05:54.465595 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:54.466115 13799 external_mini_cluster.cc:1428] Setting key fce762925c6162f616157e7c9827eed1
I20260927 07:05:54.473887 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:54.467995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39975 (local address 127.13.121.254:41929)
0927 07:05:54.468869 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:54.468884 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:54.468932 (+ 48us) server_negotiation.cc:407] Connection header received
0927 07:05:54.470564 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:54.470590 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:54.470973 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:54.471370 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:54.472367 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:54.472374 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:54.472387 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:54.472400 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:54.472460 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:54.472479 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:54.472644 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:54.472747 (+ 103us) server_negotiation.cc:299] Negotiation successful
0927 07:05:54.472929 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":402,"threads_started":1}
I20260927 07:05:54.477434 3659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f: Bootstrap starting.
I20260927 07:05:54.477591 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:54.481444 3659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:54.482795 3659 log.cc:824] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:54.485131 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:54.482777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42223 (local address 127.13.121.254:41929)
0927 07:05:54.482970 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:54.482976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:54.483334 (+ 358us) server_negotiation.cc:407] Connection header received
0927 07:05:54.483546 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:54.483555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:54.483625 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:54.483745 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:54.484553 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:54.484559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:54.484564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 07:05:54.484581 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:54.484624 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 07:05:54.484643 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:54.484750 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:54.484803 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 07:05:54.484858 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 07:05:54.486289 3659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f: No bootstrap required, opened a new log
I20260927 07:05:54.493642 3659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017a075500004a83b35c558a60d6d62f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41929 } }
I20260927 07:05:54.494051 3659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 07:05:54.494165 3659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017a075500004a83b35c558a60d6d62f, State: Initialized, Role: FOLLOWER
I20260927 07:05:54.494855 3659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017a075500004a83b35c558a60d6d62f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41929 } }
I20260927 07:05:54.495159 3659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 07:05:54.495301 3659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 07:05:54.495455 3659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:54.497862 3659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017a075500004a83b35c558a60d6d62f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41929 } }
I20260927 07:05:54.498404 3659 leader_election.cc:302] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 017a075500004a83b35c558a60d6d62f; no voters:
I20260927 07:05:54.499004 3659 leader_election.cc:288] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 07:05:54.499178 3669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:54.499773 3669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [term 1 LEADER]: Becoming Leader. State: Replica: 017a075500004a83b35c558a60d6d62f, State: Running, Role: LEADER
I20260927 07:05:54.500353 3669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017a075500004a83b35c558a60d6d62f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41929 } }
I20260927 07:05:54.501050 3659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:54.503363 3670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "017a075500004a83b35c558a60d6d62f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017a075500004a83b35c558a60d6d62f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41929 } } }
I20260927 07:05:54.503774 3670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:54.504417 3671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 017a075500004a83b35c558a60d6d62f. Latest consensus state: current_term: 1 leader_uuid: "017a075500004a83b35c558a60d6d62f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017a075500004a83b35c558a60d6d62f" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41929 } } }
I20260927 07:05:54.504815 3671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:54.505973 3676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:54.511647 3676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:05:54.521193 3676 catalog_manager.cc:1347] Generated new cluster ID: 68ff8d97ed9f45adb02c50a629e1fe0e
I20260927 07:05:54.521319 3676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:54.532668 3676 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:54.533259 3676 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:54.552551 3676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 017a075500004a83b35c558a60d6d62f: Generated new TSK 0
I20260927 07:05:54.553722 3676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:54.567656 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3592
W20260927 07:05:54.572082 3654 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39975
W20260927 07:05:54.573134 13799 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3592: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 07:05:54.573304 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 3592
2026-09-27T07:05:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1820 ms)
[----------] 18 tests from AuthTokenIssuingTest (33426 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:54 dist-test-slave-p6hz krb5kdc[3690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:54 dist-test-slave-p6hz krb5kdc[3690](info): set up 2 sockets
Sep 27 07:05:54 dist-test-slave-p6hz krb5kdc[3690](info): commencing operation
krb5kdc: starting...
W20260927 07:05:55.603746 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.010s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:05:55 dist-test-slave-p6hz krb5kdc[3690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:05:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:05:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:05:55.854662 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:34633
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:34633,127.13.121.253:33111,127.13.121.252:41149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:05:56.180267 3703 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:05:56.180642 3703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:56.180739 3703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:56.180850 3703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:56.188359 3703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:56.188614 3703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:56.188758 3703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:56.188830 3703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:56.188918 3703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:56.199571 3703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:34633,127.13.121.253:33111,127.13.121.252:41149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:34633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:56.200905 3703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:56.202250 3703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:56.214196 3711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:56.214143 3709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:56.214166 3708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:56.214973 3703 server_base.cc:1034] running on GCE node
I20260927 07:05:56.215812 3703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:56.217212 3703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:56.218447 3703 hybrid_clock.cc:648] HybridClock initialized: now 1790492756218388 us; error 29 us; skew 500 ppm
I20260927 07:05:56.218776 3703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:56 dist-test-slave-p6hz krb5kdc[3690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:56.221895 3703 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:05:56.223574 3703 webserver.cc:460] Webserver started at http://127.13.121.254:35909/ using document root <none> and password file <none>
I20260927 07:05:56.224303 3703 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:56.224412 3703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:56.224730 3703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:56.227855 3703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "00f7c8bf498d49f59a1987f6d6e49235"
format_stamp: "Formatted at 2026-09-27 07:05:56 on dist-test-slave-p6hz"
server_key: "55a2cb262c57f833bdc87f70037ca977"
server_key_iv: "19e2284c9bf41009d83c197c5a350001"
server_key_version: "encryptionkey@0"
I20260927 07:05:56.228663 3703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "00f7c8bf498d49f59a1987f6d6e49235"
format_stamp: "Formatted at 2026-09-27 07:05:56 on dist-test-slave-p6hz"
server_key: "55a2cb262c57f833bdc87f70037ca977"
server_key_iv: "19e2284c9bf41009d83c197c5a350001"
server_key_version: "encryptionkey@0"
I20260927 07:05:56.234081 3703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 07:05:56.237697 3718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:56.239161 3703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260927 07:05:56.239343 3703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "00f7c8bf498d49f59a1987f6d6e49235"
format_stamp: "Formatted at 2026-09-27 07:05:56 on dist-test-slave-p6hz"
server_key: "55a2cb262c57f833bdc87f70037ca977"
server_key_iv: "19e2284c9bf41009d83c197c5a350001"
server_key_version: "encryptionkey@0"
I20260927 07:05:56.239562 3703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:56.272877 3703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:56.277391 3703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:56.277663 3703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:56.295595 3703 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:34633
I20260927 07:05:56.295630 3769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:34633 every 8 connection(s)
I20260927 07:05:56.296953 3703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:05:56.298522 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 3703
I20260927 07:05:56.298734 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:05:56.299512 13799 external_mini_cluster.cc:1428] Setting key 7f88e10c067dd21997e2555a2956835d
I20260927 07:05:56.301659 3770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:56.310374 3770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.121.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.253@KRBTEST.COM" created.
Sep 27 07:05:56 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:56.336725 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.315166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.13.121.254:34633)
0927 07:05:56.315943 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:56.315952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:56.316050 (+ 98us) server_negotiation.cc:407] Connection header received
0927 07:05:56.317266 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:56.317272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:56.317530 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:56.317710 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:56.319023 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.320253 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.321624 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.322194 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.322367 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:56.328738 (+ 6371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:56.328750 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:56.328771 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:56.328813 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:56.332262 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:56.333115 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:56.333125 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:56.333129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:56.333325 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:56.333766 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:56.333770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:56.333774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:56.334441 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:56.334655 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:56.335282 (+ 627us) server_negotiation.cc:299] Negotiation successful
0927 07:05:56.335710 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":209,"server-negotiator.queue_time_us":581,"thread_start_us":332,"threads_started":1}
I20260927 07:05:56.336776 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.313861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34633 (local address 127.0.0.1:60432)
0927 07:05:56.314663 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.314707 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:56.316878 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:56.317753 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:56.317761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:56.318481 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:56.318817 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:56.318835 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.320398 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.320402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:56.321492 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:56.321505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.322417 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.322424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:56.322676 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:56.323459 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:56.323485 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:56.328424 (+ 4939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:56.332473 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:56.332486 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:56.332497 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:56.332943 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:56.333480 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:56.333483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:56.333486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:56.333661 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:56.334625 (+ 964us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:56.334632 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:56.334839 (+ 207us) client_negotiation.cc:769] Sending connection context
0927 07:05:56.335072 (+ 233us) client_negotiation.cc:240] Negotiation successful
0927 07:05:56.335559 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":1,"thread_start_us":333,"threads_started":1}
I20260927 07:05:56.340129 3770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:05:56.341360 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.340864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40496)
0927 07:05:56.341118 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.341239 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260927 07:05:56.341719 3721 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
I20260927 07:05:56.343415 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.342959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40502)
0927 07:05:56.343198 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.343274 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260927 07:05:56.343631 3770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:33111: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab.
Entry for principal kudu/127.13.121.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab.
I20260927 07:05:56.360791 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.253:33111
--webserver_interface=127.13.121.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab
--principal=kudu/127.13.121.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:34633,127.13.121.253:33111,127.13.121.252:41149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:05:56.392979 3770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } attempt: 1
I20260927 07:05:56.394136 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.393695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40512)
0927 07:05:56.393881 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.393997 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260927 07:05:56.395064 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.394740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40516)
0927 07:05:56.394863 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.394931 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 07:05:56.395251 3770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:33111: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
I20260927 07:05:56.463497 3770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } attempt: 2
I20260927 07:05:56.464792 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.464291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40528)
0927 07:05:56.464553 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.464647 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260927 07:05:56.465888 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.465515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40536)
0927 07:05:56.465666 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.465750 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260927 07:05:56.466104 3770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:33111: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
I20260927 07:05:56.557364 3770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } attempt: 3
I20260927 07:05:56.558531 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.558110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40542)
0927 07:05:56.558304 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.558397 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 07:05:56.559559 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.559210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40552)
0927 07:05:56.559348 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.559419 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 07:05:56.559729 3770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:33111: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
W20260927 07:05:56.686430 3777 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:05:56.686816 3777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:56.686921 3777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:56.687031 3777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:56.694442 3777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:56.694554 3777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:56.694639 3777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:56.694708 3777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:56.694789 3777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:56.703004 3770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } attempt: 4
I20260927 07:05:56.704198 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.703644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40568)
0927 07:05:56.703950 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.704040 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260927 07:05:56.705228 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.704843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40582)
0927 07:05:56.704992 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.705076 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 07:05:56.705279 3777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:34633,127.13.121.253:33111,127.13.121.252:41149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab
--principal=kudu/127.13.121.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.253:33111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.121.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
W20260927 07:05:56.705391 3770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:33111: Network error: Client connection negotiation failed: client connection to 127.13.121.253:33111: connect: Connection refused (error 111)
I20260927 07:05:56.706621 3777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:56.707985 3777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:56.720664 3783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:56.720784 3785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:56.720623 3782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:56.721829 3777 server_base.cc:1034] running on GCE node
I20260927 07:05:56.722687 3777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:56.724184 3777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:56.725428 3777 hybrid_clock.cc:648] HybridClock initialized: now 1790492756725328 us; error 85 us; skew 500 ppm
I20260927 07:05:56.725785 3777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:56 dist-test-slave-p6hz krb5kdc[3690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:56.728809 3777 init.cc:377] Logged in from keytab as kudu/127.13.121.253@KRBTEST.COM (short username kudu)
I20260927 07:05:56.730259 3777 webserver.cc:460] Webserver started at http://127.13.121.253:38765/ using document root <none> and password file <none>
I20260927 07:05:56.730935 3777 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:56.731036 3777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:56.731372 3777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:56.734355 3777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/data/instance:
uuid: "f1aca00517ed41e1941a6de3e6cb4983"
format_stamp: "Formatted at 2026-09-27 07:05:56 on dist-test-slave-p6hz"
server_key: "55283449878fc2c3e697034e57763373"
server_key_iv: "6daaabb54ec9d5cc698a307ed0c448ed"
server_key_version: "encryptionkey@0"
I20260927 07:05:56.735175 3777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/wal/instance:
uuid: "f1aca00517ed41e1941a6de3e6cb4983"
format_stamp: "Formatted at 2026-09-27 07:05:56 on dist-test-slave-p6hz"
server_key: "55283449878fc2c3e697034e57763373"
server_key_iv: "6daaabb54ec9d5cc698a307ed0c448ed"
server_key_version: "encryptionkey@0"
I20260927 07:05:56.740559 3777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:05:56.744714 3792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:05:56.746281 3777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 07:05:56.746455 3777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/wal
uuid: "f1aca00517ed41e1941a6de3e6cb4983"
format_stamp: "Formatted at 2026-09-27 07:05:56 on dist-test-slave-p6hz"
server_key: "55283449878fc2c3e697034e57763373"
server_key_iv: "6daaabb54ec9d5cc698a307ed0c448ed"
server_key_version: "encryptionkey@0"
I20260927 07:05:56.746680 3777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:56.773910 3777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:56.778203 3777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:56.778474 3777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:56.796185 3777 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.253:33111
I20260927 07:05:56.796208 3843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.253:33111 every 8 connection(s)
I20260927 07:05:56.797485 3777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
I20260927 07:05:56.801086 3844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:56.805101 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 3777
I20260927 07:05:56.805295 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-1/wal/instance
I20260927 07:05:56.805750 13799 external_mini_cluster.cc:1428] Setting key 7f021e63ada5e8e9ccbd29647d5c1959
I20260927 07:05:56.808282 3844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:05:56 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.252@KRBTEST.COM; defaulting to no policy
I20260927 07:05:56.829631 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.811158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.13.121.254:34633)
0927 07:05:56.811360 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:56.811367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:56.812694 (+ 1327us) server_negotiation.cc:407] Connection header received
0927 07:05:56.814626 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:56.814632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:56.814716 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:56.814817 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:56.816358 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.817294 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.818792 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.819405 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.819524 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:56.823220 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:56.823227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:56.823232 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:56.823276 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:56.826605 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:56.827466 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:56.827472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:56.827476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:56.827624 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:56.828067 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:56.828072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:56.828074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:56.828364 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:56.828493 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:56.829209 (+ 716us) server_negotiation.cc:299] Negotiation successful
0927 07:05:56.829396 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.252@KRBTEST.COM" created.
I20260927 07:05:56.830614 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.811685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34633 (local address 127.0.0.1:60434)
0927 07:05:56.812491 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.812549 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:56.814334 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:56.815043 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:56.815051 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:56.815622 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:56.816194 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:56.816212 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.817444 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.817449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:56.818652 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:56.818664 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.819595 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.819600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:56.819800 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:56.820491 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:56.820516 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:56.822911 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:56.826781 (+ 3870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:56.826789 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:56.826805 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:56.827335 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:56.827724 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:56.827727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:56.827730 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:56.827938 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:56.828502 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:56.828509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:56.828858 (+ 349us) client_negotiation.cc:769] Sending connection context
0927 07:05:56.829092 (+ 234us) client_negotiation.cc:240] Negotiation successful
0927 07:05:56.829507 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":309,"threads_started":1}
I20260927 07:05:56.832422 3844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:05:56 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:05:56.846725 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.833265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40586)
0927 07:05:56.833449 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.833467 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:56.833571 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:56.834420 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:56.834424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:56.834833 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:56.834999 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:56.835010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.836365 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.836371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:56.837505 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:56.837519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.838310 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.838320 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:56.838505 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:56.839170 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:56.839194 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:56.841080 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:56.844624 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:56.844627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:56.844630 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:56.844969 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:56.845325 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:56.845328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:56.845330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:56.845444 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:56.846047 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:56.846050 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:56.846161 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 07:05:56.846263 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 07:05:56.846457 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260927 07:05:56.847218 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.833375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.13.121.253:33111)
0927 07:05:56.834012 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:56.834018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:56.834044 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:05:56.834089 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:56.834095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:56.834269 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:56.834383 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:56.835186 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.836238 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.837636 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:56.838171 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:56.838260 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:56.841298 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:56.841312 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:56.841329 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:56.841363 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:56.844504 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:56.845099 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:56.845107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:56.845111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:56.845239 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:56.845539 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:56.845544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:56.845546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:56.845930 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:56.846145 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:56.846414 (+ 269us) server_negotiation.cc:299] Negotiation successful
0927 07:05:56.846646 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":360,"threads_started":1}
I20260927 07:05:56.849480 3844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:05:56.850608 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.850140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42340)
0927 07:05:56.850304 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.850432 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260927 07:05:56.850828 3793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
I20260927 07:05:56.852358 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.852025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42350)
0927 07:05:56.852187 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.852259 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260927 07:05:56.852633 3844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41149: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab.
Entry for principal kudu/127.13.121.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab.
I20260927 07:05:56.863416 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.252:41149
--webserver_interface=127.13.121.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab
--principal=kudu/127.13.121.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:33011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:34633,127.13.121.253:33111,127.13.121.252:41149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:05:56.901927 3844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } attempt: 1
I20260927 07:05:56.903093 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.902630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42360)
0927 07:05:56.902819 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.902941 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260927 07:05:56.904143 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.903830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42370)
0927 07:05:56.903961 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.904036 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 07:05:56.904350 3844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41149: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
I20260927 07:05:56.972573 3844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } attempt: 2
I20260927 07:05:56.973816 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.973312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42382)
0927 07:05:56.973522 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.973636 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260927 07:05:56.974773 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:56.974433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42384)
0927 07:05:56.974587 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:56.974657 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260927 07:05:56.975004 3844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41149: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
I20260927 07:05:57.004673 3770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } attempt: 5
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:05:57.018817 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.005720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.13.121.253:33111)
0927 07:05:57.005912 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.005918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.005936 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:05:57.006005 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.006011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.006116 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.006203 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.007042 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.007791 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.008924 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.009502 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.009631 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.013582 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.013588 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.013591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.013630 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.016192 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.017053 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.017056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.017059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.017178 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.017583 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.017586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.017587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.017858 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.017973 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.018410 (+ 437us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.018599 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 07:05:57.019109 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.005481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40600)
0927 07:05:57.005675 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.005695 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:57.005807 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:57.006266 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:57.006272 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:57.006675 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:57.006865 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.006874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.007927 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.007933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.008781 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.008792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.009695 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.009702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.009914 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.010622 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:57.010649 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:57.013340 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:57.016352 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.016357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.016362 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.016918 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.017287 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.017292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.017296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.017480 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.017979 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:57.017985 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:57.018155 (+ 170us) client_negotiation.cc:769] Sending connection context
0927 07:05:57.018316 (+ 161us) client_negotiation.cc:240] Negotiation successful
0927 07:05:57.018650 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260927 07:05:57.020205 3770 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:05:57.021385 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.020874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42390)
0927 07:05:57.021089 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.021200 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260927 07:05:57.022627 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.022088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42396)
0927 07:05:57.022334 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.022449 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260927 07:05:57.022897 3770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41149: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
I20260927 07:05:57.066251 3844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } attempt: 3
I20260927 07:05:57.067457 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.066905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42412)
0927 07:05:57.067192 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.067290 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260927 07:05:57.068746 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.068260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42418)
0927 07:05:57.068529 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.068618 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260927 07:05:57.068967 3844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41149: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
I20260927 07:05:57.074147 3770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } attempt: 1
I20260927 07:05:57.075629 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.074938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42432)
0927 07:05:57.075244 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.075374 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260927 07:05:57.076829 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.076404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42446)
0927 07:05:57.076579 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.076683 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260927 07:05:57.077103 3770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41149: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
I20260927 07:05:57.145329 3770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } attempt: 2
I20260927 07:05:57.146595 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.146006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42452)
0927 07:05:57.146263 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.146382 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260927 07:05:57.147675 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.147285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42462)
0927 07:05:57.147438 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.147535 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260927 07:05:57.147897 3770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41149: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
W20260927 07:05:57.196938 3851 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:05:57.197301 3851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:05:57.197423 3851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:05:57.197537 3851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:05:57.205077 3851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:05:57.205265 3851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:05:57.205363 3851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:05:57.205444 3851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:05:57.205530 3851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:05:57.212198 3844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } attempt: 4
I20260927 07:05:57.213459 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.212914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42474)
0927 07:05:57.213170 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.213270 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260927 07:05:57.214432 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.214088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42480)
0927 07:05:57.214236 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.214309 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260927 07:05:57.214643 3844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41149: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
I20260927 07:05:57.216653 3851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:33011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:34633,127.13.121.253:33111,127.13.121.252:41149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab
--principal=kudu/127.13.121.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.252:41149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.121.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:05:57.217983 3851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:05:57.219384 3851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:05:57.231467 3857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:57.231495 3856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:05:57.231392 3859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:05:57.232652 3851 server_base.cc:1034] running on GCE node
I20260927 07:05:57.233559 3851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:05:57.235124 3851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:05:57.236392 3851 hybrid_clock.cc:648] HybridClock initialized: now 1790492757236303 us; error 65 us; skew 500 ppm
I20260927 07:05:57.236739 3851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:05:57.239921 3851 init.cc:377] Logged in from keytab as kudu/127.13.121.252@KRBTEST.COM (short username kudu)
I20260927 07:05:57.241690 3851 webserver.cc:460] Webserver started at http://127.13.121.252:34895/ using document root <none> and password file <none>
I20260927 07:05:57.242393 3851 fs_manager.cc:359] Metadata directory not provided
I20260927 07:05:57.242496 3851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:05:57.242803 3851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:05:57.245890 3851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/data/instance:
uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc"
format_stamp: "Formatted at 2026-09-27 07:05:57 on dist-test-slave-p6hz"
server_key: "b027a56d7cc6c8df7b347c596cfdfede"
server_key_iv: "4ea17a71ab57e13534c9a070ad75fc84"
server_key_version: "encryptionkey@0"
I20260927 07:05:57.246729 3851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/wal/instance:
uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc"
format_stamp: "Formatted at 2026-09-27 07:05:57 on dist-test-slave-p6hz"
server_key: "b027a56d7cc6c8df7b347c596cfdfede"
server_key_iv: "4ea17a71ab57e13534c9a070ad75fc84"
server_key_version: "encryptionkey@0"
I20260927 07:05:57.252350 3851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260927 07:05:57.254130 3770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } attempt: 3
I20260927 07:05:57.255290 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.254767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42494)
0927 07:05:57.254976 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.255080 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 07:05:57.256347 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.255953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42496)
0927 07:05:57.256160 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.256237 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260927 07:05:57.256382 3866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260927 07:05:57.256654 3770 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41149: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41149: connect: Connection refused (error 111)
I20260927 07:05:57.257869 3851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260927 07:05:57.258036 3851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/wal
uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc"
format_stamp: "Formatted at 2026-09-27 07:05:57 on dist-test-slave-p6hz"
server_key: "b027a56d7cc6c8df7b347c596cfdfede"
server_key_iv: "4ea17a71ab57e13534c9a070ad75fc84"
server_key_version: "encryptionkey@0"
I20260927 07:05:57.258275 3851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:05:57.273581 3851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:05:57.277742 3851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:05:57.277993 3851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:05:57.295416 3851 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.252:41149
I20260927 07:05:57.295434 3917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.252:41149 every 8 connection(s)
I20260927 07:05:57.296730 3851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
I20260927 07:05:57.297457 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 3851
I20260927 07:05:57.297648 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790492406663968-13799-0/minicluster-data/master-2/wal/instance
I20260927 07:05:57.298205 13799 external_mini_cluster.cc:1428] Setting key 9a0d8f4756ece2f5511e567346d7d4f4
I20260927 07:05:57.301245 3918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:05:57.311538 3918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:05:57.327690 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.305946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.13.121.252:41149)
0927 07:05:57.306737 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.306752 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.306801 (+ 49us) server_negotiation.cc:407] Connection header received
0927 07:05:57.308465 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.308500 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.309040 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.309504 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.310745 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.312702 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.314367 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.315304 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.315492 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.319828 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.319836 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.319846 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.319883 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.323059 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.323936 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.323944 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.323954 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.324162 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.324493 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.324497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.324501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.325218 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.325498 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.325946 (+ 448us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.326362 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":327,"threads_started":1}
I20260927 07:05:57.330739 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.314739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:34633 (local address 127.0.0.1:60446)
0927 07:05:57.315644 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.315692 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:57.315831 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:57.316209 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:57.316217 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:57.317017 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:57.317338 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.317352 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.318337 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.318342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.319455 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.319468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.320266 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.320273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.320471 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.320966 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:57.320989 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:57.323744 (+ 2755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:57.326951 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.326965 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.326973 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.327557 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.328939 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.328942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.328944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.329077 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.329699 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:57.329704 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:57.329900 (+ 196us) client_negotiation.cc:769] Sending connection context
0927 07:05:57.330098 (+ 198us) client_negotiation.cc:240] Negotiation successful
0927 07:05:57.330354 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":498,"threads_started":1}
I20260927 07:05:57.330790 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.314693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60446 (local address 127.13.121.254:34633)
0927 07:05:57.314886 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.314892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.315774 (+ 882us) server_negotiation.cc:407] Connection header received
0927 07:05:57.315953 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.315959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.316038 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.316133 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.317483 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.318186 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.319599 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.320086 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.320178 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.323963 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.323970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.323974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.324016 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.326823 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.327691 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.327695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.327698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.328792 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.329195 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.329200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.329203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.329578 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.329730 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.330237 (+ 507us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.330439 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 07:05:57.332525 3918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:05:57.346115 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.330689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.13.121.254:34633)
0927 07:05:57.330999 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.331005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.331025 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:05:57.331199 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.331205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.331281 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.331386 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.332471 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.333422 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.334890 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.335512 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.335608 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.340054 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.340059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.340063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.340097 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.343283 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.344097 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.344102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.344104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.344259 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.344623 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.344628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.344631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.344941 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.345250 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.345579 (+ 329us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.345763 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":742,"threads_started":1}
I20260927 07:05:57.346346 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.333147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:33111 (local address 127.0.0.1:40616)
0927 07:05:57.333309 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.333328 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:57.333419 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:57.333754 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:57.333758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:57.334179 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:57.334361 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.334372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.335524 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.335528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.336754 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.336770 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.337486 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.337493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.337700 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.338184 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:57.338206 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:57.340270 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:57.343717 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.343722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.343726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.344268 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.344668 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.344671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.344673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.344807 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.345409 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:57.345414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:57.345597 (+ 183us) client_negotiation.cc:769] Sending connection context
0927 07:05:57.345749 (+ 152us) client_negotiation.cc:240] Negotiation successful
0927 07:05:57.346031 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 07:05:57.346771 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.333230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40616 (local address 127.13.121.253:33111)
0927 07:05:57.333400 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.333405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.333423 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:05:57.333574 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.333580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.333642 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.333708 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.334507 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.335393 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.336902 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.337317 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.337404 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.340505 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.340512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.340517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.340560 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.343568 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.344389 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.344394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.344397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.344556 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.344911 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.344916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.344919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.345256 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.345383 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.345858 (+ 475us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.346064 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:05:57.347661 3918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492757, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:05:57.360737 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.348456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42510)
0927 07:05:57.348602 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.348619 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:57.348707 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:57.349061 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:57.349064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:57.349397 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:57.349532 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.349539 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.350526 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.350532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.351625 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.351636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.352439 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.352445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.352658 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.353220 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:57.353243 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:57.355420 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:57.358356 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.358361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.358365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.358712 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.359026 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.359030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.359034 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.359185 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.359679 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:57.359684 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:57.359826 (+ 142us) client_negotiation.cc:769] Sending connection context
0927 07:05:57.359945 (+ 119us) client_negotiation.cc:240] Negotiation successful
0927 07:05:57.360328 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 07:05:57.360805 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.348644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42510 (local address 127.13.121.252:41149)
0927 07:05:57.348813 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.348818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.348836 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:05:57.348882 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.348886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.348953 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.349050 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.349664 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.350413 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.351752 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.352262 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.352354 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.355579 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.355584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.355587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.355617 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.358236 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.358812 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.358818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.358821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.358933 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.359277 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.359282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.359285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.359558 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.359691 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.360090 (+ 399us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.360462 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":55}
I20260927 07:05:57.362661 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.347901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40624 (local address 127.13.121.253:33111)
0927 07:05:57.348060 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.348065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.348128 (+ 63us) server_negotiation.cc:407] Connection header received
0927 07:05:57.348280 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.348286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.348344 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.348416 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.349849 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.350663 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.351954 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.352414 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.352531 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.356922 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.356928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.356931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.356964 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.359528 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.360274 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.360280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.360283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.360450 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.361261 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.361265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.361267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.361575 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.361849 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.362112 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.362350 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 07:05:57.366461 3918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc: Bootstrap starting.
I20260927 07:05:57.367813 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:05:57.370373 3918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:57.371589 3918 log.cc:824] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:57.371224 3745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "00f7c8bf498d49f59a1987f6d6e49235"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60460
I20260927 07:05:57.374432 3918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc: No bootstrap required, opened a new log
I20260927 07:05:57.381580 3918 raft_consensus.cc:348] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } }
I20260927 07:05:57.381934 3918 raft_consensus.cc:729] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bccf2f2cc4c8491bb08895ea3c72f7fc, State: Initialized, Role: FOLLOWER
I20260927 07:05:57.382366 3918 consensus_queue.cc:260] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } }
I20260927 07:05:57.383275 3925 sys_catalog.cc:455] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.383584 3925 sys_catalog.cc:458] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:05:57.384383 3918 sys_catalog.cc:564] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [sys.catalog]: configured and running, proceeding with master startup.
W20260927 07:05:57.393622 3936 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:05:57.393778 3936 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:05:57.391407 3851 master_runner.cc:186] Error getting master registration for 127.13.121.254:34633: OK, instance_id { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" instance_seqno: 1790492756290453 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:05:57.402341 3851 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:05:57.433857 3770 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } attempt: 4
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
I20260927 07:05:57.449019 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.434667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42526)
0927 07:05:57.434886 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.434907 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:57.435053 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:57.435431 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:57.435436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:57.435829 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:57.436007 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.436017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.437065 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.437069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.438013 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.438024 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.438752 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.438758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.438942 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.439636 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:57.439660 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:57.442135 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:57.446153 (+ 4018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.446159 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.446164 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.446712 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.447117 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.447122 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.447126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.447334 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.447993 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:57.447997 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:57.448166 (+ 169us) client_negotiation.cc:769] Sending connection context
0927 07:05:57.448322 (+ 156us) client_negotiation.cc:240] Negotiation successful
0927 07:05:57.448625 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260927 07:05:57.449054 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.434821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42526 (local address 127.13.121.252:41149)
0927 07:05:57.435083 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.435089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.435111 (+ 22us) server_negotiation.cc:407] Connection header received
0927 07:05:57.435214 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.435220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.435291 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.435372 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.436157 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.436917 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.438146 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.438615 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.438721 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.442340 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.442345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.442348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.442388 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.445987 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.446840 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.446847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.446851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.447001 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.447442 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.447447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.447450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.447880 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.448057 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.448440 (+ 383us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.448695 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260927 07:05:57.454965 3770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235: Bootstrap starting.
I20260927 07:05:57.459120 3770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:57.460744 3770 log.cc:824] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:57.464280 3770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235: No bootstrap required, opened a new log
I20260927 07:05:57.472065 3770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } }
I20260927 07:05:57.472522 3770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f7c8bf498d49f59a1987f6d6e49235, State: Initialized, Role: FOLLOWER
I20260927 07:05:57.473050 3770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } }
I20260927 07:05:57.474174 3939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.474613 3939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:05:57.475518 3770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:57.475678 3745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 07:05:57.475890 3745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:57.479322 3745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } }
I20260927 07:05:57.481871 3893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00f7c8bf498d49f59a1987f6d6e49235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc"
I20260927 07:05:57.482366 3893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:57.482585 3745 leader_election.cc:288] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [CANDIDATE]: Term 1 election: Requested vote from peers f1aca00517ed41e1941a6de3e6cb4983 (127.13.121.253:33111), bccf2f2cc4c8491bb08895ea3c72f7fc (127.13.121.252:41149)
I20260927 07:05:57.486052 3893 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f7c8bf498d49f59a1987f6d6e49235 in term 1.
I20260927 07:05:57.486748 3719 leader_election.cc:302] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f7c8bf498d49f59a1987f6d6e49235, bccf2f2cc4c8491bb08895ea3c72f7fc; no voters:
I20260927 07:05:57.487304 3939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:05:57.497789 3939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [term 1 LEADER]: Becoming Leader. State: Replica: 00f7c8bf498d49f59a1987f6d6e49235, State: Running, Role: LEADER
I20260927 07:05:57.498508 3939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } }
I20260927 07:05:57.506371 3945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00f7c8bf498d49f59a1987f6d6e49235. Latest consensus state: current_term: 1 leader_uuid: "00f7c8bf498d49f59a1987f6d6e49235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.506840 3945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:57.511927 3950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:05:57.513834 3844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } attempt: 5
Sep 27 07:05:57 dist-test-slave-p6hz krb5kdc[3690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492756, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
I20260927 07:05:57.526599 3950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260927 07:05:57.530158 3872 tablet.cc:2369] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 07:05:57.530668 3724 tablet.cc:2369] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:05:57.534693 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.514547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41149 (local address 127.0.0.1:42542)
0927 07:05:57.514963 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0927 07:05:57.514983 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 07:05:57.515130 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:05:57.516105 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:05:57.516112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:05:57.516675 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:05:57.516898 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.516914 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.518501 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.518507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.520084 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:05:57.520104 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.521415 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.521423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:05:57.521674 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.522443 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:05:57.522469 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:05:57.525518 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:05:57.531409 (+ 5891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.531414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.531418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.532016 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.532549 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:05:57.532555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:05:57.532560 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:05:57.532786 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:05:57.533554 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:05:57.533560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:05:57.533755 (+ 195us) client_negotiation.cc:769] Sending connection context
0927 07:05:57.533900 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 07:05:57.534259 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260927 07:05:57.535797 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.515391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42542 (local address 127.13.121.252:41149)
0927 07:05:57.515583 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.515589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.515609 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:05:57.515680 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.515686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.515771 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.515866 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.517119 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.518309 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.520281 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.521046 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.521170 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.525787 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.525797 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.525802 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.525854 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.531220 (+ 5366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.532197 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.532205 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.532209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.532405 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.532928 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.532934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.532937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.533330 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.533482 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.535108 (+ 1626us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.535392 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 07:05:57.542151 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983: Bootstrap starting.
I20260927 07:05:57.546838 3844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983: Neither blocks nor log segments found. Creating new log.
I20260927 07:05:57.548159 3893 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [term 1 FOLLOWER]: Refusing update from remote peer 00f7c8bf498d49f59a1987f6d6e49235: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:57.548270 3844 log.cc:824] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983: Log is configured to *not* fsync() on all Append() calls
I20260927 07:05:57.549374 3945 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:57.563879 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983: No bootstrap required, opened a new log
I20260927 07:05:57.602756 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.568146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.13.121.254:34633)
0927 07:05:57.568677 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.568684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.582189 (+ 13505us) server_negotiation.cc:407] Connection header received
0927 07:05:57.582745 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.582752 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.582867 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.582943 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.585426 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.586515 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.588558 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.589297 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.589405 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.595639 (+ 6234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.595646 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.595651 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.595703 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.599310 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.600210 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.600216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.600220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.600416 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.600919 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.600925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.600928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.601319 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.601565 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.602159 (+ 594us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.602405 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385}
I20260927 07:05:57.604575 3925 sys_catalog.cc:455] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00f7c8bf498d49f59a1987f6d6e49235. Latest consensus state: current_term: 1 leader_uuid: "00f7c8bf498d49f59a1987f6d6e49235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.604938 3925 sys_catalog.cc:458] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:05:57.606361 3953 mvcc.cc:205] Tried to move back new op lower bound from 7333858334904037376 to 7333858334720401408. Current Snapshot: MvccSnapshot[applied={T|T < 7333858334904037376}]
I20260927 07:05:57.613548 3956 mvcc.cc:205] Tried to move back new op lower bound from 7333858334904037376 to 7333858334720401408. Current Snapshot: MvccSnapshot[applied={T|T < 7333858334904037376}]
I20260927 07:05:57.615447 3844 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } }
I20260927 07:05:57.617466 3844 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1aca00517ed41e1941a6de3e6cb4983, State: Initialized, Role: FOLLOWER
I20260927 07:05:57.620222 3844 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } }
I20260927 07:05:57.626363 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.568349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.13.121.253:33111)
0927 07:05:57.568827 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.568833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.593490 (+ 24657us) server_negotiation.cc:407] Connection header received
0927 07:05:57.593771 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.593778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.593868 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.593943 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.595744 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.596832 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.611330 (+ 14498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.612068 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.612183 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.617043 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.617051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.617056 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.617107 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.621159 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.622058 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.622064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.622068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.622381 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.624480 (+ 2099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.624485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.624488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.624896 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.625151 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.625746 (+ 595us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.626020 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358}
I20260927 07:05:57.626688 3925 sys_catalog.cc:455] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00f7c8bf498d49f59a1987f6d6e49235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.626950 3925 sys_catalog.cc:458] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:05:57.627631 3939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00f7c8bf498d49f59a1987f6d6e49235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.627951 3939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:57.630698 3950 catalog_manager.cc:1347] Generated new cluster ID: 86714df58b7b47d0b47ac9e825b6311c
I20260927 07:05:57.630816 3950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:05:57.641055 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.568967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42550 (local address 127.13.121.252:41149)
0927 07:05:57.569792 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.569798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.599196 (+ 29398us) server_negotiation.cc:407] Connection header received
0927 07:05:57.599478 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.599485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.599595 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.599673 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.601494 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.602516 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.618611 (+ 16095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.619434 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.619560 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.631332 (+ 11772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:05:57.631339 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:05:57.631345 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:05:57.631391 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:05:57.635088 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.636046 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.636051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.636055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.636211 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:05:57.639256 (+ 3045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:05:57.639262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:05:57.639265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:05:57.639634 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:05:57.639862 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.640402 (+ 540us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.640651 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698}
I20260927 07:05:57.647066 3819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00f7c8bf498d49f59a1987f6d6e49235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f1aca00517ed41e1941a6de3e6cb4983"
I20260927 07:05:57.647419 3818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:05:57.644625 3966 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.648074 3966 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:05:57.651081 3818 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [term 1 FOLLOWER]: Refusing update from remote peer 00f7c8bf498d49f59a1987f6d6e49235: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:05:57.652784 3950 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:05:57.654451 3819 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f7c8bf498d49f59a1987f6d6e49235 in term 1.
I20260927 07:05:57.654764 3950 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:05:57.658262 3939 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:05:57.659687 3844 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:05:57.677611 3703 master_runner.cc:186] Error getting master registration for 127.13.121.253:33111: OK, instance_id { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" instance_seqno: 1790492756791172 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:05:57.701023 3703 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:05:57.705709 3950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235: Generated new TSK 0
I20260927 07:05:57.707609 3950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:05:57.743032 3973 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00f7c8bf498d49f59a1987f6d6e49235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.743526 3973 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:05:57.743861 3966 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00f7c8bf498d49f59a1987f6d6e49235. Latest consensus state: current_term: 1 leader_uuid: "00f7c8bf498d49f59a1987f6d6e49235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.744107 3966 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:05:57.745824 3945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00f7c8bf498d49f59a1987f6d6e49235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f7c8bf498d49f59a1987f6d6e49235" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 34633 } } peers { permanent_uuid: "f1aca00517ed41e1941a6de3e6cb4983" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 33111 } } peers { permanent_uuid: "bccf2f2cc4c8491bb08895ea3c72f7fc" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41149 } } }
I20260927 07:05:57.746109 3945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f7c8bf498d49f59a1987f6d6e49235 [sys.catalog]: This master's current role is: LEADER
I20260927 07:05:57.764809 3985 catalog_manager.cc:1259] Loaded cluster ID: 86714df58b7b47d0b47ac9e825b6311c
I20260927 07:05:57.765077 3985 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983: loading cluster ID for follower catalog manager: success
I20260927 07:05:57.768800 3985 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983: acquiring CA information for follower catalog manager: success
I20260927 07:05:57.781733 3985 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f1aca00517ed41e1941a6de3e6cb4983: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 07:05:57.867084 3848 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:57.837914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.13.121.253:33111)
0927 07:05:57.838149 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.838155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.865136 (+ 26981us) server_negotiation.cc:407] Connection header received
0927 07:05:57.865580 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.865587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.865689 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.865762 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.866835 (+ 1073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57970: BlockingRecv error: recv got EOF from 127.0.0.1:57970 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260927 07:05:57.869402 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:57.846221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55536 (local address 127.13.121.252:41149)
0927 07:05:57.846416 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.846422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.867704 (+ 21282us) server_negotiation.cc:407] Connection header received
0927 07:05:57.868095 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.868101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.868178 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.868256 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.869148 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55536: BlockingRecv error: recv got EOF from 127.0.0.1:55536 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260927 07:05:57.871564 3774 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:57.831709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.13.121.254:34633)
0927 07:05:57.831913 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.831919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.862687 (+ 30768us) server_negotiation.cc:407] Connection header received
0927 07:05:57.862906 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.862912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.863009 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.863129 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.871304 (+ 8175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35586: BlockingRecv error: recv got EOF from 127.0.0.1:35586 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 07:05:57.921036 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:57.904662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57980 (local address 127.13.121.253:33111)
0927 07:05:57.904871 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.904878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.912590 (+ 7712us) server_negotiation.cc:407] Connection header received
0927 07:05:57.913163 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.913170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.913393 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.913537 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:57.914263 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.915462 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.917888 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:57.918606 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:57.918713 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:57.919526 (+ 813us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:57.920231 (+ 705us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:57.920352 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:57.920570 (+ 218us) server_negotiation.cc:299] Negotiation successful
0927 07:05:57.920675 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260927 07:05:57.968166 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:57.954978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55548 (local address 127.13.121.252:41149)
0927 07:05:57.955286 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:57.955293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:57.965865 (+ 10572us) server_negotiation.cc:407] Connection header received
0927 07:05:57.966486 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:57.966493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:57.966654 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:57.966738 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:57.967910 (+ 1172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55548: BlockingRecv error: recv got EOF from 127.0.0.1:55548 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
I20260927 07:05:58.018220 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.003422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57996 (local address 127.13.121.253:33111)
0927 07:05:58.003637 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.003644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.011051 (+ 7407us) server_negotiation.cc:407] Connection header received
0927 07:05:58.011648 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.011655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.011752 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.011818 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.012527 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.013733 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.015846 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.016577 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.016676 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.017318 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.017483 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.017556 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.017783 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.017876 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260927 07:05:58.031117 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:58.027858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.13.121.252:41149)
0927 07:05:58.028055 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.028062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.028580 (+ 518us) server_negotiation.cc:407] Connection header received
0927 07:05:58.028765 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.028771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.028872 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.028991 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:58.029820 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55556: BlockingRecv error: recv got EOF from 127.0.0.1:55556 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 07:05:58.044317 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.037079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.13.121.253:33111)
0927 07:05:58.037263 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.037270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.037776 (+ 506us) server_negotiation.cc:407] Connection header received
0927 07:05:58.037965 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.037972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.038081 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.038194 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.038641 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.039877 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.041877 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.042576 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.042724 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.043270 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.043480 (+ 210us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.043580 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.043738 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.043834 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260927 07:05:58.059935 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:58.057598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55562 (local address 127.13.121.252:41149)
0927 07:05:58.057795 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.057802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.058337 (+ 535us) server_negotiation.cc:407] Connection header received
0927 07:05:58.058546 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.058552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.058662 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.058789 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:58.059674 (+ 885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55562: BlockingRecv error: recv got EOF from 127.0.0.1:55562 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 07:05:58.078018 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.069128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.13.121.253:33111)
0927 07:05:58.069352 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.069368 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.071746 (+ 2378us) server_negotiation.cc:407] Connection header received
0927 07:05:58.071929 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.071935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.072049 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.072167 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.072612 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.073790 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.075799 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.076486 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.076617 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.077233 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.077370 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.077463 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.077637 (+ 174us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.077728 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
W20260927 07:05:58.088033 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:58.085307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.13.121.252:41149)
0927 07:05:58.085575 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.085581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.086398 (+ 817us) server_negotiation.cc:407] Connection header received
0927 07:05:58.086576 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.086583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.086699 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.086806 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:58.087799 (+ 993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55572: BlockingRecv error: recv got EOF from 127.0.0.1:55572 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
I20260927 07:05:58.119342 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.113598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.13.121.253:33111)
0927 07:05:58.113807 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.113818 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.114323 (+ 505us) server_negotiation.cc:407] Connection header received
0927 07:05:58.114528 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.114533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.114613 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.114689 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.115113 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.115949 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.117501 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.117996 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.118112 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.118545 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.118688 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.118768 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.118913 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.118997 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260927 07:05:58.140025 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:58.137869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55588 (local address 127.13.121.252:41149)
0927 07:05:58.138097 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.138105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.138618 (+ 513us) server_negotiation.cc:407] Connection header received
0927 07:05:58.138810 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.138816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.138899 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.138984 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:58.139803 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55588: BlockingRecv error: recv got EOF from 127.0.0.1:55588 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 07:05:58.176757 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.171367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58032 (local address 127.13.121.253:33111)
0927 07:05:58.171550 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.171557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.171922 (+ 365us) server_negotiation.cc:407] Connection header received
0927 07:05:58.172117 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.172123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.172199 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.172285 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.172723 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.173559 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.174857 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.175390 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.175519 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.176048 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.176171 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.176255 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.176397 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.176488 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260927 07:05:58.190028 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:58.187925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.13.121.252:41149)
0927 07:05:58.188180 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.188188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.188716 (+ 528us) server_negotiation.cc:407] Connection header received
0927 07:05:58.188907 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.188913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.188993 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.189082 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:58.189825 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55590: BlockingRecv error: recv got EOF from 127.0.0.1:55590 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:05:58.231789 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.225775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58046 (local address 127.13.121.253:33111)
0927 07:05:58.225973 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.225979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.226262 (+ 283us) server_negotiation.cc:407] Connection header received
0927 07:05:58.226422 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.226427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.226564 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.226710 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.227056 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.228224 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.229638 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.230385 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.230544 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.230924 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.231069 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.231196 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.231351 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.231446 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260927 07:05:58.243880 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:58.241255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55592 (local address 127.13.121.252:41149)
0927 07:05:58.241446 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.241452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.242007 (+ 555us) server_negotiation.cc:407] Connection header received
0927 07:05:58.242282 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.242289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.242424 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.242513 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:58.243625 (+ 1112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55592: BlockingRecv error: recv got EOF from 127.0.0.1:55592 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:05:58.316754 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.311018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.13.121.253:33111)
0927 07:05:58.311243 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.311250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.311612 (+ 362us) server_negotiation.cc:407] Connection header received
0927 07:05:58.311799 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.311805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.311882 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.311959 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.312371 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.313104 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.314403 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.315047 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.315392 (+ 345us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.315938 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.316100 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.316163 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.316389 (+ 226us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.316479 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260927 07:05:58.326217 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:05:58.324416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55598 (local address 127.13.121.252:41149)
0927 07:05:58.324616 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.324622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.324947 (+ 325us) server_negotiation.cc:407] Connection header received
0927 07:05:58.325113 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.325117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.325204 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.325284 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:05:58.326013 (+ 729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55598: BlockingRecv error: recv got EOF from 127.0.0.1:55598 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 07:05:58.396060 3936 catalog_manager.cc:1259] Loaded cluster ID: 86714df58b7b47d0b47ac9e825b6311c
I20260927 07:05:58.396163 3936 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc: loading cluster ID for follower catalog manager: success
I20260927 07:05:58.398628 3936 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc: acquiring CA information for follower catalog manager: success
I20260927 07:05:58.400554 3936 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bccf2f2cc4c8491bb08895ea3c72f7fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:05:58.462920 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.457821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.13.121.253:33111)
0927 07:05:58.458060 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.458066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.458374 (+ 308us) server_negotiation.cc:407] Connection header received
0927 07:05:58.458467 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.458471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.458547 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.458634 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.459009 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.459711 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.461073 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.461596 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.461694 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.462141 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.462263 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.462333 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.462546 (+ 213us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.462621 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:05:58.474429 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.468677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.13.121.252:41149)
0927 07:05:58.468881 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.468887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.469190 (+ 303us) server_negotiation.cc:407] Connection header received
0927 07:05:58.469395 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.469399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.469574 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.469680 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.470088 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.470822 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.472194 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.472619 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.472736 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.473253 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.473792 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.473930 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.474035 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.474128 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 07:05:58.492695 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.480861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55616 (local address 127.13.121.252:41149)
0927 07:05:58.481320 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.481327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.486688 (+ 5361us) server_negotiation.cc:407] Connection header received
0927 07:05:58.487027 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.487032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.487115 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.487203 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.488130 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.489152 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.490775 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.491320 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.491450 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.491932 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.492042 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.492091 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.492330 (+ 239us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.492429 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285}
I20260927 07:05:58.493155 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.481089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.13.121.253:33111)
0927 07:05:58.481494 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.481500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.487183 (+ 5683us) server_negotiation.cc:407] Connection header received
0927 07:05:58.487337 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.487342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.487432 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.487528 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.488245 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.488954 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.491116 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.491699 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.491809 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.492501 (+ 692us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.492601 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.492674 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.492804 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.492901 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256}
I20260927 07:05:58.493484 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:05:58.480595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35598 (local address 127.13.121.254:34633)
0927 07:05:58.486749 (+ 6154us) server_negotiation.cc:206] Beginning negotiation
0927 07:05:58.486754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:05:58.486772 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:05:58.486906 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:05:58.486913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:05:58.487133 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:05:58.487324 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:05:58.487980 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.489062 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.491187 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:05:58.491685 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:05:58.491812 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:05:58.492288 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:05:58.492733 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:05:58.492833 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 07:05:58.493054 (+ 221us) server_negotiation.cc:299] Negotiation successful
0927 07:05:58.493140 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6036,"thread_start_us":5919,"threads_started":1}
I20260927 07:05:58.498840 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3703
I20260927 07:05:58.499871 3765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:58.611116 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 3703
W20260927 07:05:58.628758 3796 connection.cc:535] server connection from 127.0.0.1:40600 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 07:05:58.629276 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3777
I20260927 07:05:58.630395 3839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:58.739637 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 3777
I20260927 07:05:58.758396 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 3851
I20260927 07:05:58.759534 3913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:05:58.865877 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 3851
2026-09-27T07:05:58Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4296 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 07:05:58 dist-test-slave-p6hz krb5kdc[4098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 07:05:58 dist-test-slave-p6hz krb5kdc[4098](info): set up 2 sockets
Sep 27 07:05:58 dist-test-slave-p6hz krb5kdc[4098](info): commencing operation
krb5kdc: starting...
W20260927 07:05:59.895783 13799 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/test-user.keytab.
Sep 27 07:06:00 dist-test-slave-p6hz krb5kdc[4098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T07:06:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T07:06:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.121.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
Entry for principal kudu/127.13.121.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab.
I20260927 07:06:00.154958 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41575
--webserver_interface=127.13.121.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:37947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41575,127.13.121.253:42639,127.13.121.252:41317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 07:06:00.484359 4111 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:06:00.484724 4111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:06:00.484817 4111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:06:00.484929 4111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:06:00.492288 4111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:06:00.492400 4111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:06:00.492480 4111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:06:00.492545 4111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:06:00.492630 4111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:06:00.502715 4111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:37947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41575,127.13.121.253:42639,127.13.121.252:41317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.254.keytab
--principal=kudu/127.13.121.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.254:41575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.121.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:06:00.504081 4111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:06:00.505357 4111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:06:00.517100 4116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:06:00.517086 4117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:06:00.517141 4119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:06:00.518473 4111 server_base.cc:1034] running on GCE node
I20260927 07:06:00.519423 4111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:06:00.520943 4111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:06:00.522173 4111 hybrid_clock.cc:648] HybridClock initialized: now 1790492760522110 us; error 41 us; skew 500 ppm
I20260927 07:06:00.522511 4111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:06:00 dist-test-slave-p6hz krb5kdc[4098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492760, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:06:00.529930 4111 init.cc:377] Logged in from keytab as kudu/127.13.121.254@KRBTEST.COM (short username kudu)
I20260927 07:06:00.531524 4111 webserver.cc:460] Webserver started at http://127.13.121.254:39329/ using document root <none> and password file <none>
I20260927 07:06:00.532284 4111 fs_manager.cc:359] Metadata directory not provided
I20260927 07:06:00.532385 4111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:06:00.532773 4111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:06:00.536048 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/data/instance:
uuid: "3a7bf602e0ca49479c6b6ff5583caaf0"
format_stamp: "Formatted at 2026-09-27 07:06:00 on dist-test-slave-p6hz"
server_key: "9f8ad277574e23b15c656ad4e2117ea0"
server_key_iv: "429e081d65871c01341f900d769d7d1b"
server_key_version: "encryptionkey@0"
I20260927 07:06:00.536938 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/wal/instance:
uuid: "3a7bf602e0ca49479c6b6ff5583caaf0"
format_stamp: "Formatted at 2026-09-27 07:06:00 on dist-test-slave-p6hz"
server_key: "9f8ad277574e23b15c656ad4e2117ea0"
server_key_iv: "429e081d65871c01341f900d769d7d1b"
server_key_version: "encryptionkey@0"
I20260927 07:06:00.543040 4111 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 07:06:00.547029 4126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:06:00.548641 4111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 07:06:00.548820 4111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/wal
uuid: "3a7bf602e0ca49479c6b6ff5583caaf0"
format_stamp: "Formatted at 2026-09-27 07:06:00 on dist-test-slave-p6hz"
server_key: "9f8ad277574e23b15c656ad4e2117ea0"
server_key_iv: "429e081d65871c01341f900d769d7d1b"
server_key_version: "encryptionkey@0"
I20260927 07:06:00.549058 4111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:06:00.569607 4111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:06:00.573814 4111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:06:00.574090 4111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:06:00.591754 4111 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.254:41575
I20260927 07:06:00.591761 4177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.254:41575 every 8 connection(s)
I20260927 07:06:00.593114 4111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/data/info.pb
I20260927 07:06:00.596848 4178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:06:00.599725 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 4111
I20260927 07:06:00.599882 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-0/wal/instance
I20260927 07:06:00.600276 13799 external_mini_cluster.cc:1428] Setting key b5a0f85d7d64099b764f40fec83b548a
I20260927 07:06:00.604054 4178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:06:00 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492760, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.253@KRBTEST.COM" created.
I20260927 07:06:00.629248 4182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.607916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46586 (local address 127.13.121.254:41575)
0927 07:06:00.609246 (+ 1330us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:00.609254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:00.609304 (+ 50us) server_negotiation.cc:407] Connection header received
0927 07:06:00.611116 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:00.611124 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:00.611493 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:00.611695 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:00.612992 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:00.614660 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:00.616156 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:00.617018 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:00.617196 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:00.620930 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:00.620942 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:00.620960 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:00.621005 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:00.624738 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:00.625571 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:00.625578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:00.625581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:00.625738 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:00.626272 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:00.626276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:00.626278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:00.626838 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:00.627005 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:00.627691 (+ 686us) server_negotiation.cc:299] Negotiation successful
0927 07:06:00.628086 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":1110,"spinlock_wait_cycles":46848,"thread_start_us":890,"threads_started":1}
I20260927 07:06:00.629282 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.607926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41575 (local address 127.0.0.1:46586)
0927 07:06:00.608873 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.608936 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0927 07:06:00.610727 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:06:00.611736 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:06:00.611741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:06:00.612333 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:06:00.612695 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:00.612729 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:00.614818 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:00.614821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:00.616011 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:00.616023 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:00.617254 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:00.617259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:00.617456 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:00.618084 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:06:00.618107 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:06:00.620650 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:06:00.624930 (+ 4280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:00.624964 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:00.624975 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:00.625428 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:00.625902 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:00.625906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:00.625909 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:00.626150 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:00.627038 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:06:00.627047 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:06:00.627318 (+ 271us) client_negotiation.cc:769] Sending connection context
0927 07:06:00.627539 (+ 221us) client_negotiation.cc:240] Negotiation successful
0927 07:06:00.628223 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":209,"thread_start_us":361,"threads_started":1}
I20260927 07:06:00.632151 4178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:06:00.633347 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.632930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46188)
0927 07:06:00.633116 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.633227 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260927 07:06:00.633646 4130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
I20260927 07:06:00.635296 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.634861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46194)
0927 07:06:00.635016 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.635092 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260927 07:06:00.635514 4178 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:42639: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab.
Entry for principal kudu/127.13.121.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab.
I20260927 07:06:00.659945 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.253:42639
--webserver_interface=127.13.121.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab
--principal=kudu/127.13.121.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:37947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41575,127.13.121.253:42639,127.13.121.252:41317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:06:00.684877 4178 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } attempt: 1
I20260927 07:06:00.686143 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.685626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46206)
0927 07:06:00.685849 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.686011 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260927 07:06:00.687168 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.686783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46214)
0927 07:06:00.686936 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.687006 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260927 07:06:00.687343 4178 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:42639: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
I20260927 07:06:00.755575 4178 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } attempt: 2
I20260927 07:06:00.756676 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.756211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46226)
0927 07:06:00.756446 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.756542 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260927 07:06:00.757627 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.757252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46236)
0927 07:06:00.757403 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.757489 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 07:06:00.757802 4178 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:42639: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
I20260927 07:06:00.849076 4178 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } attempt: 3
I20260927 07:06:00.850447 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.849814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46244)
0927 07:06:00.850101 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.850202 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260927 07:06:00.851552 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.851125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46256)
0927 07:06:00.851300 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.851384 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 07:06:00.851758 4178 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:42639: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
W20260927 07:06:00.992744 4185 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:06:00.993110 4185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:06:00.993203 4185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:06:00.993312 4185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260927 07:06:00.994998 4178 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } attempt: 4
I20260927 07:06:00.996335 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.995705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46272)
0927 07:06:00.996050 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.996165 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260927 07:06:00.997594 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:00.997077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46280)
0927 07:06:00.997220 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:00.997318 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 07:06:00.997852 4178 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.253:42639: Network error: Client connection negotiation failed: client connection to 127.13.121.253:42639: connect: Connection refused (error 111)
W20260927 07:06:01.001971 4185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:06:01.002086 4185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:06:01.002177 4185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:06:01.002238 4185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:06:01.002319 4185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:06:01.012770 4185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:37947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41575,127.13.121.253:42639,127.13.121.252:41317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.253.keytab
--principal=kudu/127.13.121.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.253:42639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.121.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:06:01.014209 4185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 07:06:01.015594 4185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:06:01.027609 4191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:06:01.027626 4193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:06:01.027571 4190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:06:01.033005 4185 server_base.cc:1034] running on GCE node
I20260927 07:06:01.033941 4185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:06:01.035516 4185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:06:01.036783 4185 hybrid_clock.cc:648] HybridClock initialized: now 1790492761036707 us; error 45 us; skew 500 ppm
I20260927 07:06:01.037133 4185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:06:01.040604 4185 init.cc:377] Logged in from keytab as kudu/127.13.121.253@KRBTEST.COM (short username kudu)
I20260927 07:06:01.042299 4185 webserver.cc:460] Webserver started at http://127.13.121.253:42701/ using document root <none> and password file <none>
I20260927 07:06:01.043112 4185 fs_manager.cc:359] Metadata directory not provided
I20260927 07:06:01.043260 4185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:06:01.043587 4185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:06:01.046685 4185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/data/instance:
uuid: "8d60bbe22f52446792bcba3134d6a148"
format_stamp: "Formatted at 2026-09-27 07:06:01 on dist-test-slave-p6hz"
server_key: "59ea46ec84d4809f9ca333e1cf43cb95"
server_key_iv: "de45100ed91c039684f2dbfe0e1a502f"
server_key_version: "encryptionkey@0"
I20260927 07:06:01.047546 4185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/wal/instance:
uuid: "8d60bbe22f52446792bcba3134d6a148"
format_stamp: "Formatted at 2026-09-27 07:06:01 on dist-test-slave-p6hz"
server_key: "59ea46ec84d4809f9ca333e1cf43cb95"
server_key_iv: "de45100ed91c039684f2dbfe0e1a502f"
server_key_version: "encryptionkey@0"
I20260927 07:06:01.052981 4185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:06:01.057010 4200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:06:01.058552 4185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:06:01.058756 4185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/wal
uuid: "8d60bbe22f52446792bcba3134d6a148"
format_stamp: "Formatted at 2026-09-27 07:06:01 on dist-test-slave-p6hz"
server_key: "59ea46ec84d4809f9ca333e1cf43cb95"
server_key_iv: "de45100ed91c039684f2dbfe0e1a502f"
server_key_version: "encryptionkey@0"
I20260927 07:06:01.058979 4185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:06:01.074451 4185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:06:01.078856 4185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:06:01.079159 4185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:06:01.098532 4185 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.253:42639
I20260927 07:06:01.098474 4251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.253:42639 every 8 connection(s)
I20260927 07:06:01.099999 4185 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/data/info.pb
I20260927 07:06:01.104189 4252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:06:01.105162 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 4185
I20260927 07:06:01.105324 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-1/wal/instance
I20260927 07:06:01.105762 13799 external_mini_cluster.cc:1428] Setting key 73c06cc6aefeaab5b68919cbe569e1bf
I20260927 07:06:01.111097 4252 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.121.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.121.252@KRBTEST.COM" created.
I20260927 07:06:01.135762 4182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.114135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46598 (local address 127.13.121.254:41575)
0927 07:06:01.114356 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.114363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.115882 (+ 1519us) server_negotiation.cc:407] Connection header received
0927 07:06:01.118473 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.118479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.118572 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.118668 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.120904 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.121649 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.124066 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.124541 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.124660 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.129313 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.129319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.129322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.129364 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.132086 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.132996 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.133002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.133005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.133149 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.133719 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.133724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.133727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.134097 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.134244 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.135285 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.135504 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 07:06:01.137120 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.114688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41575 (local address 127.0.0.1:46598)
0927 07:06:01.115682 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.115732 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 07:06:01.118059 (+ 2327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:06:01.118970 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:06:01.118982 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:06:01.119828 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:06:01.120638 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.120665 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.121833 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.121838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.123840 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.123860 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.124798 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.124806 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.125134 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.125927 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:06:01.125964 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:06:01.128954 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:06:01.132273 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.132285 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.132304 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.132829 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.133273 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.133279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.133283 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.133551 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.134255 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:06:01.134265 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:06:01.134791 (+ 526us) client_negotiation.cc:769] Sending connection context
0927 07:06:01.135115 (+ 324us) client_negotiation.cc:240] Negotiation successful
0927 07:06:01.135727 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":400,"threads_started":1}
I20260927 07:06:01.139026 4252 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:06:01.156008 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.140010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46292)
0927 07:06:01.140233 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.140254 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 07:06:01.140387 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:06:01.141407 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:06:01.141416 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:06:01.141883 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:06:01.142065 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.142077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.143698 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.143703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.144566 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.144579 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.145809 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.145823 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.146145 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.146735 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:06:01.146753 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:06:01.149065 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:06:01.153114 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.153119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.153123 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.153653 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.154119 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.154122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.154125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.154273 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.155050 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:06:01.155053 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:06:01.155271 (+ 218us) client_negotiation.cc:769] Sending connection context
0927 07:06:01.155400 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 07:06:01.155639 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260927 07:06:01.156507 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.140220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46292 (local address 127.13.121.253:42639)
0927 07:06:01.140938 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.140945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.140973 (+ 28us) server_negotiation.cc:407] Connection header received
0927 07:06:01.141036 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.141042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.141238 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.141363 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.142238 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.143549 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.144719 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.145561 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.145711 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.149329 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.149348 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.149368 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.149407 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.152927 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.153830 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.153840 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.153844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.154005 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.154383 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.154388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.154392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.154885 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.155234 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.155587 (+ 353us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.155874 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":410,"threads_started":1}
I20260927 07:06:01.158906 4252 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:06:01.160130 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.159626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47302)
0927 07:06:01.159815 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.159932 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260927 07:06:01.160382 4204 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
I20260927 07:06:01.162135 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.161661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47310)
0927 07:06:01.161831 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.161939 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260927 07:06:01.162397 4252 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41317: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.121.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab.
Entry for principal kudu/127.13.121.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab.
I20260927 07:06:01.169958 13799 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8NDO8F/build/asan/bin/kudu
/tmp/dist-test-task8NDO8F/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.252:41317
--webserver_interface=127.13.121.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab
--principal=kudu/127.13.121.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.121.212:37947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.121.254:41575,127.13.121.253:42639,127.13.121.252:41317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 07:06:01.211781 4252 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } attempt: 1
I20260927 07:06:01.213070 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.212539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47322)
0927 07:06:01.212764 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.212874 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260927 07:06:01.214148 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.213751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47334)
0927 07:06:01.213916 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.213992 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 07:06:01.214416 4252 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41317: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
I20260927 07:06:01.282704 4252 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } attempt: 2
I20260927 07:06:01.283921 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.283481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47350)
0927 07:06:01.283671 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.283765 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260927 07:06:01.284929 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.284605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47366)
0927 07:06:01.284732 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.284814 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 07:06:01.285158 4252 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41317: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
I20260927 07:06:01.297111 4178 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } attempt: 5
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492760, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:06:01.311317 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.298030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46300 (local address 127.13.121.253:42639)
0927 07:06:01.298191 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.298196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.298216 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:06:01.298287 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.298291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.298371 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.298454 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.299437 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.300143 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.301882 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.302306 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.302405 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.306139 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.306145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.306150 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.306189 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.308705 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.309516 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.309520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.309522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.309656 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.310037 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.310040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.310042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.310334 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.310445 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.310870 (+ 425us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.311045 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 07:06:01.311491 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.297788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46300)
0927 07:06:01.298023 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.298046 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 07:06:01.298179 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:06:01.298510 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:06:01.298515 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:06:01.299014 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:06:01.299282 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.299296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.300289 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.300295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.301708 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.301728 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.302500 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.302506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.302716 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.303390 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:06:01.303422 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:06:01.305924 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:06:01.308864 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.308870 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.308874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.309390 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.309767 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.309772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.309775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.309945 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.310449 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:06:01.310454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:06:01.310631 (+ 177us) client_negotiation.cc:769] Sending connection context
0927 07:06:01.310802 (+ 171us) client_negotiation.cc:240] Negotiation successful
0927 07:06:01.311120 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260927 07:06:01.312685 4178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:06:01.313645 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.313278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47374)
0927 07:06:01.313436 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.313513 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260927 07:06:01.314566 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.314223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47376)
0927 07:06:01.314392 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.314463 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260927 07:06:01.314769 4178 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41317: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
I20260927 07:06:01.365983 4178 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } attempt: 1
I20260927 07:06:01.367308 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.366780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47392)
0927 07:06:01.366995 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.367093 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260927 07:06:01.368341 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.368013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47398)
0927 07:06:01.368159 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.368232 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 07:06:01.368559 4178 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41317: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
I20260927 07:06:01.376382 4252 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } attempt: 3
I20260927 07:06:01.377614 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.377130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47412)
0927 07:06:01.377339 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.377456 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260927 07:06:01.378809 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.378320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47416)
0927 07:06:01.378596 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.378686 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
W20260927 07:06:01.379169 4252 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41317: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
I20260927 07:06:01.436794 4178 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } attempt: 2
I20260927 07:06:01.438030 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.437559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47418)
0927 07:06:01.437772 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.437865 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260927 07:06:01.439088 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.438717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47422)
0927 07:06:01.438893 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.438971 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260927 07:06:01.439306 4178 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41317: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
W20260927 07:06:01.504037 4259 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 07:06:01.504379 4259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 07:06:01.504448 4259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 07:06:01.504530 4259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 07:06:01.511986 4259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 07:06:01.512095 4259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 07:06:01.512151 4259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 07:06:01.512204 4259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 07:06:01.512264 4259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 07:06:01.522413 4252 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } attempt: 4
I20260927 07:06:01.523515 4259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.121.212:37947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.121.254:41575,127.13.121.253:42639,127.13.121.252:41317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/krb5kdc/kudu_127.13.121.252.keytab
--principal=kudu/127.13.121.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.121.252:41317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.121.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 05:59:20 UTC on ccycloud.quasar-zavphu.root.comops.site
build id 4675737
ASAN enabled
I20260927 07:06:01.523706 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.523185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47438)
0927 07:06:01.523435 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.523528 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260927 07:06:01.524780 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.524438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47448)
0927 07:06:01.524588 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.524662 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 07:06:01.524919 4259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260927 07:06:01.525018 4252 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41317: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
I20260927 07:06:01.526319 4259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 07:06:01.538841 4264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:06:01.538802 4265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 07:06:01.538865 4267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 07:06:01.539867 4259 server_base.cc:1034] running on GCE node
I20260927 07:06:01.540896 4259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 07:06:01.542598 4259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 07:06:01.543864 4259 hybrid_clock.cc:648] HybridClock initialized: now 1790492761543764 us; error 69 us; skew 500 ppm
I20260927 07:06:01.544276 4259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 07:06:01.545552 4178 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } attempt: 3
I20260927 07:06:01.546725 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.546202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47452)
0927 07:06:01.546454 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.546543 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 07:06:01.547895 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.547491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47466)
0927 07:06:01.547689 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.547777 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260927 07:06:01.548108 4178 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.121.252:41317: Network error: Client connection negotiation failed: client connection to 127.13.121.252:41317: connect: Connection refused (error 111)
I20260927 07:06:01.548164 4259 init.cc:377] Logged in from keytab as kudu/127.13.121.252@KRBTEST.COM (short username kudu)
I20260927 07:06:01.549726 4259 webserver.cc:460] Webserver started at http://127.13.121.252:37669/ using document root <none> and password file <none>
I20260927 07:06:01.550463 4259 fs_manager.cc:359] Metadata directory not provided
I20260927 07:06:01.550558 4259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 07:06:01.550837 4259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 07:06:01.554239 4259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/data/instance:
uuid: "fc83a864351f4404bcf27a84b18ea7bb"
format_stamp: "Formatted at 2026-09-27 07:06:01 on dist-test-slave-p6hz"
server_key: "f8b0a0362bf07942537dcc8adbefc435"
server_key_iv: "142546fe3671cc01e057a5e1fcba1143"
server_key_version: "encryptionkey@0"
I20260927 07:06:01.555073 4259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/wal/instance:
uuid: "fc83a864351f4404bcf27a84b18ea7bb"
format_stamp: "Formatted at 2026-09-27 07:06:01 on dist-test-slave-p6hz"
server_key: "f8b0a0362bf07942537dcc8adbefc435"
server_key_iv: "142546fe3671cc01e057a5e1fcba1143"
server_key_version: "encryptionkey@0"
I20260927 07:06:01.560909 4259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 07:06:01.564920 4274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 07:06:01.566470 4259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 07:06:01.566650 4259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/data,/tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/wal
uuid: "fc83a864351f4404bcf27a84b18ea7bb"
format_stamp: "Formatted at 2026-09-27 07:06:01 on dist-test-slave-p6hz"
server_key: "f8b0a0362bf07942537dcc8adbefc435"
server_key_iv: "142546fe3671cc01e057a5e1fcba1143"
server_key_version: "encryptionkey@0"
I20260927 07:06:01.566875 4259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 07:06:01.582155 4259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 07:06:01.586597 4259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 07:06:01.586872 4259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 07:06:01.605465 4259 rpc_server.cc:307] RPC server started. Bound to: 127.13.121.252:41317
I20260927 07:06:01.605500 4325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.121.252:41317 every 8 connection(s)
I20260927 07:06:01.606833 4259 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/data/info.pb
I20260927 07:06:01.610654 4326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 07:06:01.614641 13799 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8NDO8F/build/asan/bin/kudu as pid 4259
I20260927 07:06:01.614840 13799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8NDO8F/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790492406663968-13799-0/minicluster-data/master-2/wal/instance
I20260927 07:06:01.615391 13799 external_mini_cluster.cc:1428] Setting key d29a8a1c01da53687957e6a0f1c5ee1f
I20260927 07:06:01.618646 4326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:06:01.637058 4182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.622646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46600 (local address 127.13.121.254:41575)
0927 07:06:01.622844 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.622849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.623634 (+ 785us) server_negotiation.cc:407] Connection header received
0927 07:06:01.623814 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.623820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.623892 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.623973 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.625466 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.626278 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.627968 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.628368 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.628469 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.631742 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.631746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.631749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.631783 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.634292 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.634970 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.634974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.634976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.635112 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.635532 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.635536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.635538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.635797 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.635944 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.636471 (+ 527us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.636702 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 07:06:01.638027 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.617413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47472 (local address 127.13.121.252:41317)
0927 07:06:01.618179 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.618192 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.618234 (+ 42us) server_negotiation.cc:407] Connection header received
0927 07:06:01.619861 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.619885 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.620342 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.620708 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.622410 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.624352 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.626022 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.626861 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.626998 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.631363 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.631371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.631387 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.631428 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.634314 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.634949 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.634955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.634957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.635101 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.635459 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.635463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.635465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.635977 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.636244 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.636563 (+ 319us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.637034 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":194,"server-negotiator.queue_time_us":562,"thread_start_us":343,"threads_started":1}
I20260927 07:06:01.638109 4330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.622712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.254:41575 (local address 127.0.0.1:46600)
0927 07:06:01.623492 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.623529 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 07:06:01.623697 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:06:01.624134 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:06:01.624156 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:06:01.624947 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:06:01.625279 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.625292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.626466 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.626469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.627829 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.627844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.628598 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.628607 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.628791 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.629338 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:06:01.629373 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:06:01.631565 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:06:01.634427 (+ 2862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.634439 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.634452 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.634827 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.635250 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.635255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.635259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.635417 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.635927 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:06:01.635939 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:06:01.636134 (+ 195us) client_negotiation.cc:769] Sending connection context
0927 07:06:01.636353 (+ 219us) client_negotiation.cc:240] Negotiation successful
0927 07:06:01.636851 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":310,"threads_started":1}
I20260927 07:06:01.639649 4326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.254@KRBTEST.COM
I20260927 07:06:01.654248 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.640438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46314 (local address 127.13.121.253:42639)
0927 07:06:01.640628 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.640634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.640653 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:06:01.640746 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.640751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.640832 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.640908 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.641751 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.642807 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.644496 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.645149 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.645284 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.648479 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.648487 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.648493 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.648539 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.651513 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.652301 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.652306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.652308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.652464 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.652898 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.652902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.652904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.653168 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.653324 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.653774 (+ 450us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.653965 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 07:06:01.654321 4330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.640342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.253:42639 (local address 127.0.0.1:46314)
0927 07:06:01.640504 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.640522 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:06:01.640638 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:06:01.640953 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:06:01.640958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:06:01.641370 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:06:01.641568 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.641592 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.642967 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.642971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.644347 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.644361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.645282 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.645286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.645429 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.646093 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:06:01.646117 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:06:01.648275 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:06:01.651653 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.651657 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.651661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.652194 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.652597 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.652602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.652606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.652786 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.653302 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:06:01.653306 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:06:01.653480 (+ 174us) client_negotiation.cc:769] Sending connection context
0927 07:06:01.653686 (+ 206us) client_negotiation.cc:240] Negotiation successful
0927 07:06:01.653983 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260927 07:06:01.655483 4326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } has no permanent_uuid. Determining permanent_uuid...
I20260927 07:06:01.657063 4182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.641338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46608 (local address 127.13.121.254:41575)
0927 07:06:01.641510 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.641516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.641535 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:06:01.641591 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.641609 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.641704 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.641788 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.643219 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.644319 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.645953 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.646572 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.646705 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.650880 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.650885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.650889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.650924 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.654101 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.654901 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.654906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.654909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.655076 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.655481 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.655486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.655489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.655850 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.656110 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.656538 (+ 428us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.656772 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.252@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.121.253@KRBTEST.COM
I20260927 07:06:01.668481 4330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.656191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47476)
0927 07:06:01.656351 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.656369 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 07:06:01.656472 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:06:01.656737 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:06:01.656741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:06:01.657219 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:06:01.657383 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.657394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.658441 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.658446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.659599 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.659611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.660229 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.660234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.660425 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.660946 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:06:01.660963 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:06:01.662867 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:06:01.665987 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.665991 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.665995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.666354 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.666797 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.666801 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.666803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.666937 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.667518 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:06:01.667523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:06:01.667657 (+ 134us) client_negotiation.cc:769] Sending connection context
0927 07:06:01.667785 (+ 128us) client_negotiation.cc:240] Negotiation successful
0927 07:06:01.668079 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260927 07:06:01.668510 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.656243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.13.121.252:41317)
0927 07:06:01.656373 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.656379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.656413 (+ 34us) server_negotiation.cc:407] Connection header received
0927 07:06:01.656562 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.656567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.656637 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.656710 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.657504 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.658339 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.659714 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.660100 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.660205 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.663046 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.663051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.663054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.663090 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.665830 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.666494 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.666500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.666503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.666662 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.667062 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.667066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.667068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.667408 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.667545 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.667871 (+ 326us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.668229 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":53}
I20260927 07:06:01.672461 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.658725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.13.121.253:42639)
0927 07:06:01.658887 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.658893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.658911 (+ 18us) server_negotiation.cc:407] Connection header received
0927 07:06:01.658983 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.658988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.659059 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.659175 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.660336 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.661001 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.662162 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.662656 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.662767 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.666973 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.666978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.666982 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.667015 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.669622 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.670234 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.670239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.670242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.670421 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.670874 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.670879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.670883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.671255 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.671516 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.671877 (+ 361us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.672120 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 07:06:01.674464 4326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb: Bootstrap starting.
I20260927 07:06:01.677613 13799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 07:06:01.678148 4326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb: Neither blocks nor log segments found. Creating new log.
I20260927 07:06:01.679455 4326 log.cc:824] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb: Log is configured to *not* fsync() on all Append() calls
I20260927 07:06:01.682717 4326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb: No bootstrap required, opened a new log
I20260927 07:06:01.680615 4153 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46608
I20260927 07:06:01.689863 4326 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } }
I20260927 07:06:01.690203 4326 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc83a864351f4404bcf27a84b18ea7bb, State: Initialized, Role: FOLLOWER
I20260927 07:06:01.690683 4326 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } }
I20260927 07:06:01.691572 4332 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.691874 4332 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:06:01.692672 4326 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [sys.catalog]: configured and running, proceeding with master startup.
W20260927 07:06:01.702014 4343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:06:01.702164 4343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:06:01.699384 4259 master_runner.cc:186] Error getting master registration for 127.13.121.254:41575: OK, instance_id { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" instance_seqno: 1790492760586655 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 07:06:01.710363 4259 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:06:01.725320 4178 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } attempt: 4
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492760, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.254@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
I20260927 07:06:01.744763 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.726230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.13.121.252:41317)
0927 07:06:01.726550 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.726557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.726583 (+ 26us) server_negotiation.cc:407] Connection header received
0927 07:06:01.726652 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.726657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.726741 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.726868 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.728193 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.729330 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.731250 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.731970 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.732111 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.737307 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.737316 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.737321 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.737366 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.740900 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.741914 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.741919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.741923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.742090 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.742705 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.742710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.742713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.743236 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.743382 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.743890 (+ 508us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.744141 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 07:06:01.744856 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.726109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47492)
0927 07:06:01.726304 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.726326 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 07:06:01.726444 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:06:01.726932 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:06:01.726938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:06:01.727662 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:06:01.728000 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.728012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.729492 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.729498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.731004 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.731021 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.732158 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.732164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.732368 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.733964 (+ 1596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:06:01.733991 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:06:01.737048 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:06:01.741113 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.741119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.741123 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.741758 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.742212 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.742218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.742222 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.742567 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.743394 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:06:01.743399 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:06:01.743619 (+ 220us) client_negotiation.cc:769] Sending connection context
0927 07:06:01.743791 (+ 172us) client_negotiation.cc:240] Negotiation successful
0927 07:06:01.744123 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260927 07:06:01.750046 4178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0: Bootstrap starting.
I20260927 07:06:01.753978 4178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0: Neither blocks nor log segments found. Creating new log.
I20260927 07:06:01.755270 4178 log.cc:824] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0: Log is configured to *not* fsync() on all Append() calls
I20260927 07:06:01.758077 4178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0: No bootstrap required, opened a new log
I20260927 07:06:01.763480 4178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } }
I20260927 07:06:01.763875 4178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7bf602e0ca49479c6b6ff5583caaf0, State: Initialized, Role: FOLLOWER
I20260927 07:06:01.764325 4178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } }
I20260927 07:06:01.765254 4346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.765632 4346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:06:01.766417 4178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:06:01.766527 4153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 07:06:01.766737 4153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:06:01.770689 4153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } }
I20260927 07:06:01.772411 4153 leader_election.cc:288] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [CANDIDATE]: Term 1 election: Requested vote from peers 8d60bbe22f52446792bcba3134d6a148 (127.13.121.253:42639), fc83a864351f4404bcf27a84b18ea7bb (127.13.121.252:41317)
I20260927 07:06:01.773375 4301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fc83a864351f4404bcf27a84b18ea7bb"
I20260927 07:06:01.773908 4301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:06:01.777514 4301 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a7bf602e0ca49479c6b6ff5583caaf0 in term 1.
I20260927 07:06:01.778952 4130 leader_election.cc:302] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7bf602e0ca49479c6b6ff5583caaf0, fc83a864351f4404bcf27a84b18ea7bb; no voters:
I20260927 07:06:01.779512 4346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 07:06:01.791522 4346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [term 1 LEADER]: Becoming Leader. State: Replica: 3a7bf602e0ca49479c6b6ff5583caaf0, State: Running, Role: LEADER
I20260927 07:06:01.793236 4346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } }
W20260927 07:06:01.795976 4357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 07:06:01.796134 4357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 07:06:01.815973 4359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a7bf602e0ca49479c6b6ff5583caaf0. Latest consensus state: current_term: 1 leader_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.817581 4359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [sys.catalog]: This master's current role is: LEADER
I20260927 07:06:01.824218 4252 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } attempt: 5
W20260927 07:06:01.827207 4132 tablet.cc:2369] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 07:06:01 dist-test-slave-p6hz krb5kdc[4098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790492761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.121.253@KRBTEST.COM for kudu/127.13.121.252@KRBTEST.COM
I20260927 07:06:01.835634 4365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 07:06:01.808525 4111 master_runner.cc:186] Error getting master registration for 127.13.121.253:42639: OK, instance_id { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" instance_seqno: 1790492761092612 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260927 07:06:01.839151 4280 tablet.cc:2369] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 07:06:01.839723 4111 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 07:06:01.842545 4365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 07:06:01.845297 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.824916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.121.252:41317 (local address 127.0.0.1:47508)
0927 07:06:01.825353 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0927 07:06:01.825377 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 07:06:01.825516 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 07:06:01.826617 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 07:06:01.826622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 07:06:01.827179 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0927 07:06:01.827401 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.827417 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.829013 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.829020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.830501 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 07:06:01.830523 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.831777 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.831783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 07:06:01.832033 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.832814 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 07:06:01.832841 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 07:06:01.835784 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 07:06:01.841878 (+ 6094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.841884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.841889 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.842533 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.843160 (+ 627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 07:06:01.843165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 07:06:01.843168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 07:06:01.843351 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 07:06:01.844249 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 07:06:01.844255 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 07:06:01.844428 (+ 173us) client_negotiation.cc:769] Sending connection context
0927 07:06:01.844593 (+ 165us) client_negotiation.cc:240] Negotiation successful
0927 07:06:01.844929 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295}
I20260927 07:06:01.846273 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.825889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.13.121.252:41317)
0927 07:06:01.826093 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.826099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.826118 (+ 19us) server_negotiation.cc:407] Connection header received
0927 07:06:01.826186 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.826191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.826294 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.826395 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.827630 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.828821 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.830710 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.831408 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.831535 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.836111 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.836120 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.836125 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.836176 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.841587 (+ 5411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.842741 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.842747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.842750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.842971 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.843622 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.843628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.843631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.844013 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.844174 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.845636 (+ 1462us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.845899 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 07:06:01.852901 4252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148: Bootstrap starting.
I20260927 07:06:01.857647 4252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148: Neither blocks nor log segments found. Creating new log.
I20260927 07:06:01.867870 4252 log.cc:824] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148: Log is configured to *not* fsync() on all Append() calls
I20260927 07:06:01.878626 4252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148: No bootstrap required, opened a new log
I20260927 07:06:01.885425 4301 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [term 1 FOLLOWER]: Refusing update from remote peer 3a7bf602e0ca49479c6b6ff5583caaf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:06:01.886818 4359 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:06:01.895196 4252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } }
I20260927 07:06:01.895675 4252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d60bbe22f52446792bcba3134d6a148, State: Initialized, Role: FOLLOWER
I20260927 07:06:01.896294 4252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } }
I20260927 07:06:01.903383 4372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.905362 4182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.860143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46622 (local address 127.13.121.254:41575)
0927 07:06:01.860386 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.860392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.871809 (+ 11417us) server_negotiation.cc:407] Connection header received
0927 07:06:01.875482 (+ 3673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.875488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.875574 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.875652 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.884998 (+ 9346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.886032 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.887849 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.888587 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.888703 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.892760 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.892767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.892771 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.892817 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.898299 (+ 5482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.899548 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.899558 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.899562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.900069 (+ 507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.900736 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.900742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.900746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.901786 (+ 1040us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.903627 (+ 1841us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.904008 (+ 381us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.904545 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 07:06:01.909891 4372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:06:01.909735 4252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 07:06:01.910182 4226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [term 0 FOLLOWER]: Advancing to term 1
I20260927 07:06:01.914422 4226 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [term 1 FOLLOWER]: Refusing update from remote peer 3a7bf602e0ca49479c6b6ff5583caaf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 07:06:01.915199 4227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8d60bbe22f52446792bcba3134d6a148"
I20260927 07:06:01.915712 4359 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 07:06:01.917328 4332 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a7bf602e0ca49479c6b6ff5583caaf0. Latest consensus state: current_term: 1 leader_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.917717 4332 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:06:01.919178 4227 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a7bf602e0ca49479c6b6ff5583caaf0 in term 1.
I20260927 07:06:01.920706 4366 mvcc.cc:205] Tried to move back new op lower bound from 7333858352601849856 to 7333858352322932736. Current Snapshot: MvccSnapshot[applied={T|T < 7333858352601849856}]
I20260927 07:06:01.922047 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.866310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46320 (local address 127.13.121.253:42639)
0927 07:06:01.866564 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.866571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.873239 (+ 6668us) server_negotiation.cc:407] Connection header received
0927 07:06:01.874014 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.874022 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.874193 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.874280 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.889636 (+ 15356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.890680 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.895382 (+ 4702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.896093 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.896200 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.904776 (+ 8576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.904785 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.904790 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.904841 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.909228 (+ 4387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.912494 (+ 3266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.912501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.912504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.912702 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.913235 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.913241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.913245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.913672 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.913916 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.914498 (+ 582us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.914786 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260927 07:06:01.923979 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:01.872029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.13.121.252:41317)
0927 07:06:01.872233 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:01.872239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:01.891740 (+ 19501us) server_negotiation.cc:407] Connection header received
0927 07:06:01.891921 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:01.891927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:01.892039 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:01.892149 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:01.899245 (+ 7096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.900286 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.902128 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:01.902924 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:01.903043 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:01.915889 (+ 12846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 07:06:01.915897 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 07:06:01.915902 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 07:06:01.915947 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 07:06:01.919518 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.920472 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.920478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.920481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.920634 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 07:06:01.921081 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 07:06:01.921086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 07:06:01.921089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 07:06:01.921459 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 07:06:01.921713 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:01.923371 (+ 1658us) server_negotiation.cc:299] Negotiation successful
0927 07:06:01.923639 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 07:06:01.926944 4332 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.927233 4332 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:06:01.933017 4359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.933317 4359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [sys.catalog]: This master's current role is: LEADER
I20260927 07:06:01.938011 4365 catalog_manager.cc:1347] Generated new cluster ID: 7630bfad98ac4ed4aef0451822e72ecd
I20260927 07:06:01.938170 4365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 07:06:01.944617 4372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a7bf602e0ca49479c6b6ff5583caaf0. Latest consensus state: current_term: 1 leader_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.944990 4372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:06:01.950191 4346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.950573 4346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0 [sys.catalog]: This master's current role is: LEADER
I20260927 07:06:01.957980 4372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7bf602e0ca49479c6b6ff5583caaf0" member_type: VOTER last_known_addr { host: "127.13.121.254" port: 41575 } } peers { permanent_uuid: "8d60bbe22f52446792bcba3134d6a148" member_type: VOTER last_known_addr { host: "127.13.121.253" port: 42639 } } peers { permanent_uuid: "fc83a864351f4404bcf27a84b18ea7bb" member_type: VOTER last_known_addr { host: "127.13.121.252" port: 41317 } } }
I20260927 07:06:01.958282 4372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 07:06:01.989744 4365 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 07:06:01.992110 4365 catalog_manager.cc:1504] Loading token signing keys...
I20260927 07:06:02.016446 4365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a7bf602e0ca49479c6b6ff5583caaf0: Generated new TSK 0
I20260927 07:06:02.017874 4365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 07:06:02.062503 4390 catalog_manager.cc:1259] Loaded cluster ID: 7630bfad98ac4ed4aef0451822e72ecd
I20260927 07:06:02.062624 4390 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148: loading cluster ID for follower catalog manager: success
I20260927 07:06:02.066525 4390 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148: acquiring CA information for follower catalog manager: success
I20260927 07:06:02.069206 4390 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8d60bbe22f52446792bcba3134d6a148: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 07:06:02.134166 13799 mini_kdc.cc:348] Time spent kdestroy: real 0.108s user 0.000s sys 0.022s
W20260927 07:06:02.188153 4182 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.171098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.13.121.254:41575)
0927 07:06:02.171350 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.171356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.184541 (+ 13185us) server_negotiation.cc:407] Connection header received
0927 07:06:02.184755 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.184762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.184859 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.185002 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.187878 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46626: BlockingRecv error: recv got EOF from 127.0.0.1:46626 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260927 07:06:02.193150 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.185297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.13.121.252:41317)
0927 07:06:02.185541 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.185547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.190622 (+ 5075us) server_negotiation.cc:407] Connection header received
0927 07:06:02.191508 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.191515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.191604 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.191690 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.192886 (+ 1196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47528: BlockingRecv error: recv got EOF from 127.0.0.1:47528 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260927 07:06:02.196727 4256 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.179783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.13.121.253:42639)
0927 07:06:02.180052 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.180059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.189170 (+ 9111us) server_negotiation.cc:407] Connection header received
0927 07:06:02.189519 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.189528 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.189819 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.189982 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.196452 (+ 6470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46324: BlockingRecv error: recv got EOF from 127.0.0.1:46324 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260927 07:06:02.245309 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.229744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.13.121.253:42639)
0927 07:06:02.229956 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.229963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.236932 (+ 6969us) server_negotiation.cc:407] Connection header received
0927 07:06:02.237549 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.237557 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.237782 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.237933 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.238649 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.239920 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.241925 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.242600 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.242704 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.243579 (+ 875us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.244331 (+ 752us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.244474 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.244702 (+ 228us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.244808 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260927 07:06:02.289744 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.280117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.13.121.252:41317)
0927 07:06:02.280308 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.280314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.287279 (+ 6965us) server_negotiation.cc:407] Connection header received
0927 07:06:02.287895 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.287902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.288057 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.288150 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.289470 (+ 1320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47538: BlockingRecv error: recv got EOF from 127.0.0.1:47538 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 07:06:02.338069 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.323398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.13.121.253:42639)
0927 07:06:02.323627 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.323633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.330180 (+ 6547us) server_negotiation.cc:407] Connection header received
0927 07:06:02.330735 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.330742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.330863 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.330936 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.331684 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.332869 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.335556 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.336263 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.336420 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.337120 (+ 700us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.337283 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.337369 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.337623 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.337723 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260927 07:06:02.378160 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.372586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.13.121.252:41317)
0927 07:06:02.372804 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.372810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.375285 (+ 2475us) server_negotiation.cc:407] Connection header received
0927 07:06:02.375486 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.375492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.375590 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.375714 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.377920 (+ 2206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47548: BlockingRecv error: recv got EOF from 127.0.0.1:47548 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 07:06:02.391861 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.384470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46352 (local address 127.13.121.253:42639)
0927 07:06:02.384675 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.384682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.385334 (+ 652us) server_negotiation.cc:407] Connection header received
0927 07:06:02.385523 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.385528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.385622 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.385732 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.386205 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.387476 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.389495 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.390211 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.390374 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.390885 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.391054 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.391239 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.391393 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.391521 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260927 07:06:02.404124 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.401713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47552 (local address 127.13.121.252:41317)
0927 07:06:02.401917 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.401924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.402559 (+ 635us) server_negotiation.cc:407] Connection header received
0927 07:06:02.402739 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.402745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.402869 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.402990 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.403877 (+ 887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47552: BlockingRecv error: recv got EOF from 127.0.0.1:47552 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 07:06:02.425827 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.412238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.13.121.253:42639)
0927 07:06:02.412495 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.412502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.419082 (+ 6580us) server_negotiation.cc:407] Connection header received
0927 07:06:02.419337 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.419344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.419458 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.419550 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.420231 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.421471 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.423524 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.424252 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.424389 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.425020 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.425179 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.425269 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.425436 (+ 167us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.425540 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
W20260927 07:06:02.439379 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.437191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47568 (local address 127.13.121.252:41317)
0927 07:06:02.437446 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.437452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.437845 (+ 393us) server_negotiation.cc:407] Connection header received
0927 07:06:02.438007 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.438013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.438113 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.438228 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.439083 (+ 855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47568: BlockingRecv error: recv got EOF from 127.0.0.1:47568 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
I20260927 07:06:02.459731 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.452683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46362 (local address 127.13.121.253:42639)
0927 07:06:02.452898 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.452912 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.453417 (+ 505us) server_negotiation.cc:407] Connection header received
0927 07:06:02.453692 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.453698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.453794 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.453893 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.454334 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.455543 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.457501 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.458242 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.458394 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.458898 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.459044 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.459193 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.459303 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.459390 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260927 07:06:02.468722 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.466244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47576 (local address 127.13.121.252:41317)
0927 07:06:02.466438 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.466444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.467165 (+ 721us) server_negotiation.cc:407] Connection header received
0927 07:06:02.467386 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.467392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.467487 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.467588 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.468454 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47576: BlockingRecv error: recv got EOF from 127.0.0.1:47576 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 07:06:02.494088 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.488136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.13.121.253:42639)
0927 07:06:02.488324 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.488330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.488676 (+ 346us) server_negotiation.cc:407] Connection header received
0927 07:06:02.488829 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.488835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.488934 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.489043 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.489408 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.490561 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.492009 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.492702 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.492847 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.493267 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.493440 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.493552 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.493693 (+ 141us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.493784 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260927 07:06:02.502549 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.500733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47588 (local address 127.13.121.252:41317)
0927 07:06:02.500937 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.500943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.501211 (+ 268us) server_negotiation.cc:407] Connection header received
0927 07:06:02.501358 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.501365 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.501505 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.501620 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.502290 (+ 670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47588: BlockingRecv error: recv got EOF from 127.0.0.1:47588 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:06:02.544790 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.538335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46378 (local address 127.13.121.253:42639)
0927 07:06:02.538601 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.538611 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.538966 (+ 355us) server_negotiation.cc:407] Connection header received
0927 07:06:02.539182 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.539189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.539324 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.539444 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.539882 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.540916 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.542732 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.543409 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.543553 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.544011 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.544133 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.544214 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.544362 (+ 148us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.544465 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260927 07:06:02.552721 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.550831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47604 (local address 127.13.121.252:41317)
0927 07:06:02.551025 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.551031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.551343 (+ 312us) server_negotiation.cc:407] Connection header received
0927 07:06:02.551522 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.551528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.551627 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.551737 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.552473 (+ 736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47604: BlockingRecv error: recv got EOF from 127.0.0.1:47604 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260927 07:06:02.625584 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.619842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.13.121.253:42639)
0927 07:06:02.620077 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.620084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.620475 (+ 391us) server_negotiation.cc:407] Connection header received
0927 07:06:02.620664 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.620668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.620740 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.620820 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.621258 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.621964 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.623990 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.624458 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.624555 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.624932 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.625037 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.625107 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.625232 (+ 125us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.625306 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260927 07:06:02.632781 4329 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 07:06:02.630967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.13.121.252:41317)
0927 07:06:02.631213 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.631218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.631484 (+ 266us) server_negotiation.cc:407] Connection header received
0927 07:06:02.631661 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.631665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.631747 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.631831 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 07:06:02.632617 (+ 786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47610: BlockingRecv error: recv got EOF from 127.0.0.1:47610 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 07:06:02.704267 4343 catalog_manager.cc:1259] Loaded cluster ID: 7630bfad98ac4ed4aef0451822e72ecd
I20260927 07:06:02.704376 4343 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb: loading cluster ID for follower catalog manager: success
I20260927 07:06:02.706947 4343 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb: acquiring CA information for follower catalog manager: success
I20260927 07:06:02.708892 4343 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fc83a864351f4404bcf27a84b18ea7bb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 07:06:02.769809 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.764522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.13.121.253:42639)
0927 07:06:02.764717 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.764722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.765081 (+ 359us) server_negotiation.cc:407] Connection header received
0927 07:06:02.765214 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.765219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.765292 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.765392 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.765884 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.766708 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.768207 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.768622 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.768731 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.769196 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.769318 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.769394 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.769507 (+ 113us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.769585 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 07:06:02.781687 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.776048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.13.121.252:41317)
0927 07:06:02.776270 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.776277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.776551 (+ 274us) server_negotiation.cc:407] Connection header received
0927 07:06:02.776737 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.776741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.776909 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.777009 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.777419 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.778124 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.779496 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.779926 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.780039 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.780577 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.781102 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.781231 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.781322 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.781413 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 07:06:02.795116 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.787678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.13.121.253:42639)
0927 07:06:02.787953 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.787960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.788930 (+ 970us) server_negotiation.cc:407] Connection header received
0927 07:06:02.789260 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.789266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.789355 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.789416 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.790207 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.791249 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.792952 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.793459 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.793578 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.794313 (+ 735us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.794436 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.794521 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.794721 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.794814 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 07:06:02.796011 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.787516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46632 (local address 127.13.121.254:41575)
0927 07:06:02.788303 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.788308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.788328 (+ 20us) server_negotiation.cc:407] Connection header received
0927 07:06:02.788451 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.788457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.788699 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.788921 (+ 222us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.789497 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.790442 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.792810 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.793501 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.793721 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.794313 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.794848 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.794993 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.795089 (+ 96us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.795246 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":563,"threads_started":1}
I20260927 07:06:02.796931 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 07:06:02.789227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.13.121.252:41317)
0927 07:06:02.789383 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 07:06:02.789389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 07:06:02.790138 (+ 749us) server_negotiation.cc:407] Connection header received
0927 07:06:02.790299 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 07:06:02.790304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 07:06:02.790394 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 07:06:02.790491 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 07:06:02.791239 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.792430 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.794605 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 07:06:02.795269 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 07:06:02.795388 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 07:06:02.796206 (+ 818us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 07:06:02.796340 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 07:06:02.796402 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 07:06:02.796597 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 07:06:02.796680 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 07:06:02.801484 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 4111
I20260927 07:06:02.802541 4173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:06:02.913218 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 4111
W20260927 07:06:02.932075 4276 connection.cc:535] server connection from 127.0.0.1:47492 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 07:06:02.936556 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 4185
I20260927 07:06:02.937795 4247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:06:03.046324 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 4185
I20260927 07:06:03.064010 13799 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8NDO8F/build/asan/bin/kudu pid 4259
I20260927 07:06:03.065158 4321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 07:06:03.173715 13799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8NDO8F/build/asan/bin/kudu with pid 4259
2026-09-27T07:06:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4309 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (8605 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (356473 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST